SFT
Why it matters
Almost every chat model you've used went through SFT. It's also the most common kind of fine-tuning that builders do themselves: when a company fine-tunes a model to answer in its house style or output a specific JSON format, that's SFT, usually with LoRA or QLoRA.
SFT is simple to set up and very effective at teaching format and behavior. It's the fastest way to get a model to reliably do a specific thing, as long as you can show it good examples.
Deep Dive
After pre-training, a language model is a powerful text predictor, but not a helpful assistant. Ask it a question and it might continue with three more questions, because that's what text on the internet often looks like. Supervised fine-tuning fixes this by showing the model many examples of the behavior you want: a prompt, and an ideal response written or approved by a person (or a stronger model). The model is trained to produce those responses, token by token.
SFT is usually the first stage of post-training. It turns a base model into something that follows instructions, uses a chat format and responds in a consistent style. Preference methods such as RLHF or DPO typically come after it.
How It Works
The training data is a set of conversations, each made of a prompt (and sometimes a system prompt and earlier turns) and a target response. The model sees the whole conversation, but the loss is usually computed only on the response tokens. This is called masking the prompt: you want the model to learn to write good answers, not to predict the user's questions.
Each example is formatted with the model's chat template, the special tokens that mark where the system, user and assistant turns begin and end. Getting the template wrong is one of the most common causes of a broken fine-tune.
Quality Beats Quantity
A recurring finding is that a small set of excellent examples beats a large set of mediocre ones. The LIMA paper (2023) showed that around 1,000 carefully chosen examples could produce a strong assistant from a good base model. In practice, a few hundred to a few thousand high-quality, diverse, consistent examples is a sensible starting point for a narrow task.
Much SFT data today is synthetic: written by a stronger model and then filtered. That's efficient, but the student inherits the teacher's habits and mistakes, so filtering matters.
SFT Versus Preference Training
SFT teaches the model to imitate. It can only be as good as its examples, and it never sees what a bad answer looks like. Preference methods (RLHF, DPO) show the model pairs of better and worse answers, which helps with subtler qualities like helpfulness, safety and tone. Reinforcement learning with verifiable rewards (RLVR) goes further for reasoning tasks by letting the model discover solutions rather than copy them. A typical modern pipeline is: pre-training, then SFT, then preference or RL training.
Common Pitfalls
- Forgetting: too many epochs on a narrow dataset can erode general skills. One to three epochs is usual.
- Inconsistent data: if examples disagree on format or tone, the model will too.
- Teaching facts by SFT: it can make the model sound sure about new information without reliably knowing it, which can increase hallucinations. Use RAG for knowledge.