Fine-tuning exists on a spectrum of how much you change:
- Light adaptation: Adjust output format or style. Needs - examples.
- Domain specialization: Teach new terminology and concepts. Needs - examples.
- Behavior transformation: Fundamentally change how the model responds. Needs + high-quality examples.
More change requires more data and compute.