TL;DR: Fine-tuning continues a pretrained model's training on your own examples, permanently shifting how it behaves. It excels at teaching style, format, and specialized skills - and it is the wrong tool for adding facts, which is retrieval's job. Try prompting first, then RAG, and fine-tune when a large example set can prove the gain.
How it works
Pretraining gives a model general language ability at a cost only large labs can pay. Fine-tuning borrows all of it: you take the finished model and run more training steps on a much smaller dataset - typically hundreds to tens of thousands of prompt-and-response pairs demonstrating exactly the behavior you want. The weights move toward your examples, and the change is baked in: unlike instructions in a prompt, fine-tuned behavior does not consume context space on every request and cannot be talked out of by a clever user message. A support team that fine-tunes on a few thousand of its best agent replies gets a model that answers in house style without being told to each time.
Full fine-tuning updates every weight, which requires serious GPU memory. Parameter-efficient methods - LoRA is the standard - freeze the original network and train small adapter matrices alongside it, capturing most of the benefit at a small fraction of the cost, cheap enough to run on a single GPU for many open-weights models. Hosted providers offer the same workflow as an API: upload examples, receive a tuned endpoint. The related technique of RLHF is also post-training, but it optimizes against preference scores rather than imitating labeled examples.
The critical judgment call is fine-tuning versus RAG, and the rule of thumb is behavior versus knowledge. Fine-tuning reliably changes how a model acts: tone, output format, domain vocabulary, following your classification taxonomy. It is a poor way to add facts - models absorb new knowledge weakly from small datasets, the facts freeze immediately, and updating means retraining. Retrieval supplies current, verifiable knowledge per request. Mature systems often use both: fine-tune for the behavior, retrieve for the facts.
Costs to respect: building a good dataset is the real work, quality beats quantity, and a sloppy set teaches sloppy behavior. Tuned models can also regress on general ability (catastrophic forgetting), and every base-model upgrade means re-tuning - so run evals before and after to prove the exercise paid for itself.
Where it sits in the AI stack
Fine-tuning sits between a general pretrained model and the specialized model you deploy:
Key tools and implementations
-
LoRA and QLoRA
Parameter-efficient methods that train small adapters instead of the whole network.
-
Hosted fine-tuning APIs
Provider services that tune a model from your uploaded examples, no GPUs to manage.
-
Axolotl and Unsloth
Open-source toolkits that streamline fine-tuning open-weights models on your own hardware.
-
Hugging Face TRL and PEFT
The standard libraries under most fine-tuning stacks, from supervised tuning to preference training.
Related entries
- RLHF (reinforcement learning from human feedback) Training a model toward answers humans prefer by scoring its outputs and reinforcing the good ones.
- Distillation Training a smaller model to imitate a larger one, keeping much of the capability at far lower cost.
- RAG (retrieval augmented generation) Fetching relevant documents at query time so a language model can answer from your data instead of memory alone.
- Open-weights models Models whose trained weights are published for anyone to download, run locally, and fine-tune.