Gains:
- Ability to recognize silent causes of degradation of the model (data drift, concept drift, upstream error) and establish three-layer (operational, input, output) monitoring
- Ability to evaluate LLM systems in multiple layers with rule checks, LLM-referee and human evaluation, and calibrate with LLM-referee human anchor
- Ability to design an eval set containing edge and security cases and turn each caught error into a permanent test case
Once a model goes into production, your work is not done; The real responsibility just begins. Because the model can silently break down when no one is looking. In this unit, we cover two complementary disciplines: evaluation (systematically measuring the quality of the model) and monitoring (constant monitoring of the model in production). Especially in LLM systems, eval is more difficult and requires more care than classical ML.
Why the production model is quietly breaking down
A bug crashes, the log prints, the alarm goes off. An ML model, on the other hand, can be wrong without causing errors. Three main causes of degradation:
- Data drift: The distribution of input data changes over time (new products, changing user behavior, seasonality). The model remains the same but the world changes.
- Concept drift: The input-output relationship changes. Fraud tactics and spam patterns evolve; What was right yesterday will be wrong today.
- Upstream corruption: A data source changes format, an area becomes free; The model silently drools with corrupted input.
Tracing is making these silent distortions audible.
What to watch: three layers
Good monitoring covers three layers:
- Operational metrics: Latency, error rate, request volume, resource usage. "Is the system standing?"
- Data/input metrics: Is the input distribution similar to that in training? Has the missing value rate increased? Have new categories arrived? “Is the model seeing familiar data?”
- Model/output metrics: Prediction distribution log? Have confidence scores dropped? And if possible, what is the accuracy compared to the ground truth? “Is the model still accurate?”
The third layer is the most valuable but the most difficult; because the real result usually comes with a delay (it becomes clear after months whether a loan will be repaid or not).
Tip: If the actual result is delayed, monitor the input and prediction distribution first. Shift of the input distribution is an early sign of accuracy degradation and can raise an alarm without waiting for the actual result.
Evaluating LLM systems: the special challenge
In classical ML, the "correct answer" is clear (class 0 or 1). The LLM outcome, on the other hand, is open-ended: there may be many correct answers to the same question, "correctness" does not fit into a single number. LLM eval approaches:
- Referenced metrics: Comparing the output to the ideal answer. Limited; because it may consider the correct answer expressed differently as "wrong".
- Rules-based checks: Is the output valid JSON? Are there any banned words? Does it contain the desired fields? Cheap, reliable, tight.
- LLM-judge (LLM-as-judge): Don't make a model ask "is this answer good according to this criterion?" It scales, but the referee itself must be verified.
- Human review: Gold standard but expensive and slow. It is used on the sample.
In practice these are used together: cheap rule checks on each output, LLM-judge on a large sample, human evaluation on a small but rigorous sample.
Weak approach / Strong approach
Weak: "LLM-I asked the referee, 92% of our answers were good. The system is great."
Güçlü: "We first human-labeled 100 printouts. We ran the LLM-judge on the same 100 printouts and measured human-judge agreement — 85% agreement, acceptable. We documented where the judge systematically went wrong (a tendency to find long answers unfairly good) and fixed his prompt. Only then did we trust the judge's scores."
The difference: the strong approach verifies the referee with a human anchor, not blindly. An unverified LLM-referee gives nice-looking but false confidence.
Attention: LLM-referee is also a model; hallucinogenic, biased (favors long/confident answers), can be inconsistent. Calibrate referee scores with human tags before making production decisions.
Evaluation set: carefully designed
A good eval set represents the variety of real usage and difficult cases. An eval filled with just easy examples will leave you in false confidence. Be sure to put it in the eval cluster:
- Edge cases: Empty input, very long input, unusual format.
- Known hard cases: Examples where the model has made mistakes in the past (as a regression test).
- Security incidents: Prompt injection attempts, malicious requests, privacy violation traps.
The eval cluster grows over time: each new bug you catch in production becomes a test case for the next evaluation.
Alarm and intervention
Monitoring remains incomplete without an alarm. There should be a threshold and a response plan for each important metric: "Notify engineer if input drift exceeds X", "Auto roll back if error rate exceeds Y". Keep alarms meaningful — too many false alarms desensitize the team and make them miss the real alarm.
three mini cases
Case 1 - Early warning. The true accuracy of a demand forecast model only became apparent at the end of the week. The team was monitoring input distribution and saw the sudden rise of a new product category on a Tuesday — something the model had never seen. They updated the model without waiting for the accuracy drop. Input monitoring saved days.
Case 2 - Unverified referee. One team reported "our quality is excellent" based on LLM-reviewer. When customer complaints increased, human monitoring was introduced: the referee counted confident but incorrect answers as "good." Once the referee was calibrated with human tags, the true quality was revealed and was much lower. Lesson: don't trust the referee without verifying it.
Case 3 - Regression testing. A prompt change resolved one issue while silently breaking another. But the team kept past bugs in the eval bucket; When the new change was tested on this cluster, the broken case was immediately caught and the change was fixed. Lesson: every fixed bug should become a permanent test case.
Copiable templates
Produce a tracking plan for this production model. Cover three layers:1) Operational (latency, error rate, volume)2) Input/data (distribution shift, missing value, new category)3) Model/output (prediction distribution, confidence, accuracy if possible)Model: [description]. How long does it take for the actual result to arrive: [duration]Add threshold and intervention recommendation for each metric.
Propose an evaluation (eval) strategy for this LLM system.Task: [description]Determine layers:- Which rule-based checks should run on each output?- What criteria should the LLM-arbitrator evaluate and how should they be validated (human anchor)?- In which sample should human evaluation be performed?List edge and safety cases that I should put in the eval set.
Check this LLM-referee prompt:- Is the evaluation criteria clear or subjective?- Is it prone to length/confidence bias?- How do I calibrate the referee with human tags?Referee prompt: [prompt]
Write a response runbook for this monitoring alarm.Alarm: [e.g. input drift threshold exceeded]Must contain: initial control steps, possible causes, rollback criteria, who to inform.
Deterioration cause table
distortion
symptom
Way to early detection
data drift
Input distribution changes
Input distribution monitoring
concept shift
Righteousness falls silently
Prediction + actual comparison
upstream error
Fields become vacant/format changes
Schema validation + missing rate
Model inconsistency
Output distribution shifts
Output distribution monitoring
Common mistakes
- Not establishing monitoring. The model breaks down silently, no one sees it.
- Track operational metrics only. The system is up, but predictions can be wrong.
- Using LLM without verifying the referee. It gives false confidence.
- Eval with easy examples. It does not indicate real difficulty.
- Not including past errors in eval. The same error returns again.
- Loud alarms. The team becomes desensitized, missing the real alarm.
In summary
The model can be inaccurate without causing errors in production; so eval and monitoring are just as important as development. Establish monitoring at three layers (operational, input, output); Use input drift as an early warning if the actual result is delayed. In LLM systems, eval is open-ended; Use rule checks, LLM-referee, and human evaluation together — but be sure to validate the LLM-referee with a human anchor. Enrich your Eval cluster with edge and security cases and turn every caught error into a permanent test case.
Application task
Write a three-layer monitoring plan for a production (or near-production) model and define threshold + alarm for at least one input-distribution metric. If you have an LLM system: tag 30 outputs with humans, run an LLM-referee on the same outputs, and measure human-referee agreement; Note the referee's systematic bias. Add at least 3 edges and 2 security cases to your eval cluster.
checklist
- [ ] Monitoring covers all three layers (operational, input, output).
- [ ] I use input drift as an early warning if the actual result is delayed.
- [ ] I calibrated the LLM-arbitrator with human labels.
- [ ] The Eval cluster contains edge and security cases.
- [ ] I turned every bug I caught into a permanent test case.
- [ ] Each important metric has a threshold and response plan.