You will be able to
- Explain what fine-tuning changes in a model and what it cannot fix
- Decide between better prompts, RAG and fine-tuning for a given problem
- Prepare and validate training data in the chat JSONL format
- Compare full fine-tuning, LoRA adapters and preference tuning in plain terms
- Evaluate a tuned model against a baseline and count the real costs
What fine-tuning changes
A pre-trained model has learned general language from a huge amount of text, and chat models are already tuned to follow instructions. Fine-tuning continues that training on a small, curated set of your examples, nudging the weights so the model's default behaviour moves towards them.
It is good at teaching how to answer: a fixed output structure, a house style, a consistent level of difficulty, the vocabulary of a narrow field, or a classification scheme with your own labels. Instructions that once needed a long prompt become the model's habit, so prompts get shorter.
It is poor at teaching what is true right now. Facts that change, such as a course timetable or this week's notes, belong in retrieval. A model tuned on old facts states them confidently long after they stop being correct, and you would need to retrain to update them.
Not every model can be tuned. Hosted providers offer fine-tuning only for some of their models, and the list changes over time, while open-weight models can always be trained if you have the hardware. Check this before you plan a project around tuning.
Prompting, RAG or fine-tuning?
Work through the options in order of cost. Improve the prompt first, with clear instructions and a few examples. If the model lacks knowledge, add RAG. Only when both are in place and a measurable gap remains does fine-tuning become worth the effort.
The lesson 15 eval set is what makes this decision honest. Score the best prompt-plus-RAG version first; that score is the baseline any tuned model must beat.
- Choose prompting when a few instructions and examples already get the behaviour you want.
- Choose RAG when answers depend on documents, private data or information that changes.
- Consider fine-tuning when you need the same format or style on every call and prompts keep slipping.
- Consider it when few-shot prompts grow so long that every request becomes slow and expensive.
- Consider it when you want a small, fast model to match a larger one on one narrow task, often by training on the larger model's checked answers, if the larger model's terms allow it.
Preparing the data
Most providers and open-source trainers accept chat examples in JSON Lines: one JSON object per line, each with a messages list of system, user and assistant turns. The assistant turn is the answer you want the model to learn, so it must be exactly the output you would be happy to ship.
Quality matters far more than volume. A few dozen consistent, carefully checked examples are enough for a first experiment, and many more will not rescue examples that contradict each other. Cover the variety you expect in real use: easy and hard topics, short and long inputs, and cases where the right answer is a polite refusal.
Use the same system prompt in training that you will use when calling the tuned model. Hold back part of the data as a validation set the model never trains on. Remove names, emails and other personal data, and make sure you have the right to use every example, especially if it came from users or another model's output.
Methods in plain terms
Supervised fine-tuning, usually shortened to SFT, learns from input and ideal-output pairs. It is the default and the right place to start.
Preference tuning, for example Direct Preference Optimization, learns from pairs where one answer is marked better than the other. It helps when you can judge answers but find it hard to write the perfect one. Reinforcement fine-tuning goes further and uses a grader that scores answers, which suits tasks with checkable results such as maths or code.
With open-weight models you also choose how many weights to train. Full fine-tuning updates all of them and needs a lot of GPU memory. LoRA freezes the original weights and trains small adapter matrices beside them; QLoRA does the same on a quantised base model. Adapters are small files, so you can keep one per task on top of a single base model.
Hosted providers run the training for you: you upload files, start a job and get a private model name back to call through the same API. For open models, libraries such as Hugging Face TRL, Unsloth and Axolotl handle the training on your own or rented GPUs.
Evaluate and count the cost
Run the tuned model on the same eval set as the baseline, with the same system prompt. Look at the pass count and also read answers on cases outside the training topics, because tuning can make a model worse at things you did not train for.
Watch training and validation loss if your tool reports them. When training loss keeps falling while validation loss rises, the model is memorising your examples rather than learning the pattern, so stop earlier or add more varied data.
Compare cost and speed per request as well as quality. A tuned model that needs a much shorter prompt can be cheaper to run than the base model with a long few-shot prompt, and that saving is often the strongest argument for tuning.
Count every cost, not just the training run. Writing and checking examples takes the most time. Hosted tuned models can cost more per token than the base model, and self-hosted ones need a server. When the provider retires the base model, you will have to repeat the work on a new one, so keep your data and scripts in version control.
Example
Validate chat JSONL and split it into train and validation files
import json, random, sys ROLES = {"system", "user", "assistant"}path = sys.argv[1] if len(sys.argv) > 1 else "quiz_examples.jsonl"examples, errors = [], []def bad(m): return not isinstance(m, dict) or m.get("role") not in ROLES or not str(m.get("content") or "").strip() for n, line in enumerate(open(path, encoding="utf-8"), 1): if not line.strip(): continue try: msgs = json.loads(line)["messages"] except (json.JSONDecodeError, KeyError, TypeError) as e: errors.append(f"line {n}: {e!r}") continue if not isinstance(msgs, list) or not msgs or any(bad(m) for m in msgs): errors.append(f"line {n}: bad role or empty content") elif msgs[-1]["role"] != "assistant": errors.append(f"line {n}: last message must come from the assistant") else: examples.append(line.strip()) if errors or not examples: sys.exit("\n".join(errors) or "No examples found.")random.Random(42).shuffle(examples)cut = max(1, len(examples) // 5) # hold back about 20% for validationopen("train.jsonl", "w", encoding="utf-8").write("\n".join(examples[cut:]) + "\n")open("valid.jsonl", "w", encoding="utf-8").write("\n".join(examples[:cut]) + "\n")print(f"OK: {len(examples) - cut} training and {cut} validation examples")The script needs no API key. It stops with a list of problems if any line is broken, and otherwise writes train.jsonl and valid.jsonl. Each provider adds its own limits on top of this basic shape, so check their documentation before uploading.
Practice
Decide whether Study Buddy's quiz style needs fine-tuning
- Write 30 lines in
quiz_examples.jsonl, each with Study Buddy's system prompt, a topic from your notes as the user turn and a perfect three-question quiz as the assistant turn. - Run the example script and fix every reported line until it prints OK with the two counts.
- Build a few-shot prompt from three of your examples and run your lesson 15 eval set with it to get a baseline score.
- If your provider or local setup supports fine-tuning, train on
train.jsonl, then run the same eval set on the tuned model. - Write one sentence in your README stating whether fine-tuning is worth it for Study Buddy, with the baseline score as evidence.
Done when: The validator prints OK with training and validation counts, you have a baseline eval score, and your README holds a dated decision on fine-tuning backed by that score.
Common mistakes
- Fine-tuning to add facts that change, instead of keeping them in a retrieval index.
- Skipping the baseline, so nobody can tell whether the tuned model actually beats a good prompt.
- Training on hundreds of inconsistent examples rather than a few dozen carefully checked ones.
- Calling the tuned model with a different system prompt from the one used in training.
Check yourself
Pick an answer to see whether it is right and why.
From the founder
Get the baseline score before you prepare a single training job. If a clear prompt with RAG already passes your eval set, I want you to write that down and move on to the next lesson.