Gains:
- Ability to distinguish whether a problem is information or behavior and evaluate fine-tuning for behavioral problems only after prompt, few-shot and RAG are exhausted.
- Understanding fine-tuning methods (SFT, LoRA/PEFT, RLHF) and data attributes that determine quality (consistency, diversity, confidentiality)
- Ability to measure the true value of fine-tuning with before-after evaluation, overlearning and catastrophic forgetting tests
Fine-tuning (customizing its behavior by training a pre-trained model further with your own data) is a powerful but expensive tool in the arsenal of the engineer working with LLMs. When used in the wrong place, it is a waste of money and time, but when used in the right place, it provides a quality that cannot be achieved otherwise. In this unit, we cover when fine-tuning is necessary, its basic methods and risks. The goal is to make you decision-making.
First the right question: is fine-tuning necessary?
The most expensive mistake of beginners is to immediately rush to fine-tuning a problem that can be solved. Set up the order like this:
- Prompt engineering: A good prompt that explains the task clearly solves most problems. Consume here first.
- Few-shot learning: Putting a few examples in the prompt shows the model the desired format and behavior.
- RAG: If the problem is "lack of information" (need for data that the model does not know), the solution is RAG (unit 4), not fine-tuning.
- Fine-tuning: It comes into play if the above is not enough and the problem is "behavior/format/style".
Key distinction: Fine-tuning is weak and risky at teaching the model new information; but it is strong at teaching how to behave (a specific format, tone, field jargon, consistent structure). “My model does not know our company information” → RAG. "Let my model always give the output in the exact format we want" → fine-tuning candidate.
Tip: Before deciding on fine-tuning, ask: "Is this a knowledge problem or a behavior problem?" Information problems are better solved with RAG, behavioral problems are better solved with fine-tuning.
Fine-tuning methods
Full fine-tuning: Retraining all parameters of the model. Most powerful but most expensive; It requires large hardware (GPU) and careful data. It's unnecessary for most teams.
Parameter-efficient fine-tuning (PEFT): Methods that freeze the vast majority of the model, training only a small set of additional parameters. The most common is LoRA (Low-Rank Adaptation: training small "adapter" layers added to the model). LoRA provides results close to full fine-tuning, with much less memory and cost; That's why it is the first choice in practice.
Supervised Fine-Tuning (SFT): Teaching the model to "respond to this input like this" with data consisting of input-ideal output pairs. It is the most common scenario.
Reinforcement learning from human feedback (RLHF): Aligning the model's behavior from people's preferred responses. It is complex and expensive; The needs of most application teams are met with SFT. It is enough to know RLHF as a concept.
Data: the heart of fine-tuning
The quality of fine-tuning depends entirely on the quality of the training data. A few hundred high quality, consistent samples are better than thousands of sloppy ones. When preparing data:
- Consistency: All examples consistently show the format and tone you want. Contradictory examples leave the model confused.
- Variety: Examples cover the variety in actual use, but are not uniform.
- Cleaning: Instances containing incorrect, biased, or hidden data are permanently passed into the model. Fine-tuning data should be read as carefully as a contract.
Caution: Every bias, error, and hidden information that enters the fine-tuning data is etched into the model and reappears in its outputs. Audit your training data as meticulously as if you were publishing it; Do not post personal data.
Review: did fine-tuning work?
Before fine-tuning, reserve a held-out eval set and measure the score of the model without fine-tuning (base model). After fine-tuning, measure again in the same bank. You can't say "it got better" without comparison. Also two traps to be aware of:
- Overlearning: Overtraining with small data causes the model to lose its ability to memorize and generalize training examples.
- Catastrophic forgetting: Overtraining on a narrow task can impair the model's overall abilities. Test whether old skills are retained while acquiring the new behavior.
Weak approach / Strong approach
Weak: "I have 3000 chat logs, let's give them all to fine-tuning, so the model can talk like us."
Güçlü: "First I evaluated the base model with 100 real tasks, noted the score. I measured how much it improved with prompts and few-shots — it wasn't enough. Then from the 3000 logs, I selected and cleaned only 400 samples with high quality, consistent format and no hidden data. I fine-tuned with LoRA, measured again with the same 100 tasks and confirmed with a separate test that the general capabilities were intact."
The difference: the strong approach exhausts alternatives first, cherry-picks data, measures before-and-after, and tests for side effects.
The reality of cost and maintenance
Fine-tuning is not a one-time job; It is a duty of care. It may be necessary to retrain when the base model is updated, needs change, or data is lost. Additionally, hosting a fine-tuned model brings additional costs and operations. Compare this total cost of ownership with the increase in quality it provides. Most of the time a good prompt + RAG is cheaper and more flexible than fine-tuning.
three mini cases
Case 1 - Unnecessary fine-tuning. A team embarked on an expensive fine-tuning project because "our model doesn't know our products." Months and budget were spent, the result was fragile — the model became obsolete every time the product catalog changed. Finally they switched to RAG: they fetched the product data from the document base, the update was instantaneous and the cost dropped. Lesson: the information problem is not solved by fine-tuning.
Case 2 - Correct fine-tuning. An insurance company wanted the model to always produce policy summaries in the same rigid structure (item by clause, with specific headings). Consistency with prompt is stuck at 70%. After LoRA fine-tuning with 300 good samples, format consistency increased to 98%. This was a behavior problem and fine-tuning was the right tool.
Case 3 - Privacy escaping into data. One team submitted the chat logs to fine-tuning without cleaning them. The logs contained real customer names and identification numbers. The fine-tuned model started "leaking" these names as output in unrelated questions. The model was withdrawn, data masked and retrained. Lesson: Hidden information in fine-tuning data is permanently transferred to the model.
Copiable templates
Help me decide if fine-tuning is necessary for this problem of mine. Problem: [description] Is this an INFORMATION problem (the model does not know something) or a BEHAVIOR problem (the model does not produce the format/tone/structure I want)? Can it be solved with prompt, few-shot and RAG first? Why try/not try each of them?Only under what condition would you recommend fine-tuning?
Check this fine-tuning dataset:1) Are the examples consistent in format and tone?2) Do they cover the variation in actual use?3) Does it contain confidential/personal data (must be masked)?4) Are there conflicting examples?A subset of the examples: [examples]List each issue and fix you found.
Produce an evaluation plan before and after fine-tuning. Task: [explanation]- How should the held-out eval set be selected?- How is the score of the base model measured?- With which metric is it compared after fine-tuning?- How do I test that general capabilities are not impaired (catastrophic forgetting)?
Suggest initial hyperparameters for fine-tuning with LoRA.Data size: [number of samples]Purpose: [format/tone teaching]Suggest epoch, learning rate, and early stopping to avoid overlearning.
Decision table: which tool when
need
try first
Fine-tuning?
The model does not know any information
RAG
no
Current data required
RAG
no
Specific rigid format
few shots
If not enough yes
Consistent tone/style
prompt + few-shot
If not enough yes
Field jargon/style
prompt
If that's not enough, LoRA
Simple task optimization
prompt engineering
Generally no
Common mistakes
- Trying to solve the information problem with fine-tuning. RAG is the right tool.
- Running into fine-tuning without consuming prompt/few-shot/RAG. Expensive and unnecessary.
- Training with low quality/conflicting data. Less but clear data is better.
- Putting confidential data into education. Permanently infiltrates the model.
- Not measuring before and after. You cannot prove recovery.
- Not testing catastrophic forgetting. The new skill may disrupt the old one.
In summary
Fine-tuning is a powerful but expensive tool and should only be considered for behavior/format problems after consuming prompt-few-shot-RAG; information problems belong to RAG. Parameter-efficient methods such as LoRA are the practical first choice. Quality depends entirely on data quality; Use small but clean, consistent and confidential data. Measure before and after, test for overlearning and loss of ability. Fine-tuning is a duty of care; Weigh its total cost against the quality it delivers.
Application task
Choose a problem and decide whether fine-tuning is necessary by distinguishing between "knowledge or behavior" and write your justification. If it is a behavior problem, prepare 20-30 consistent samples, check for hidden data, and document a before-and-after evaluation plan (held-out cluster, base score, comparison metric, forgetting test). If it can be solved with RAG/few-shot instead of fine-tuning, make a note of this too.
checklist
- [ ] I determined whether the problem is knowledge or behavior.
- [ ] I first evaluated the prompt, few-shot and RAG alternatives.
- [ ] I audited the fine-tuning data for consistency, diversity and confidentiality.
- [ ] I allocated a held-out eval cluster and measured the base score.
- [ ] I planned a before-after comparison and forgetting test.
- [ ] I compared the total cost with the quality it provides.