Gains:
- Ability to recognize the special challenges of ML related to the code-data-model trio and package and present the model online or in batch according to business need.
- Ability to implement gradual and rollback deployment patterns (shadow, canary, A/B, rollback) and add a tested rollback plan to each deployment
- Ability to keep the data-code-metric link of the model put into production traceable with evaluation threshold-controlled CI/CD and model registry
Getting a model to achieve 95% accuracy in the notebook is only half the story. The other half—often the hard part—is getting that model to real users in a reliable, scalable, and maintainable way. MLOps (Machine Learning Operations: the discipline of putting, operating, and maintaining ML models into production) combines the DevOps practices of software engineering with the unique challenges of ML. In this unit, we cover the steps of moving the model to production and how artificial intelligence helps in this process.
Why is ML different from regular software?
In ordinary software, the behavior is in the code; If the code doesn't change, the behavior doesn't change. In ML, behavior depends on both code, data and model. These three dimensions create the extra challenges of MLOps:
- Data drift: The data in production moves away from the data in training over time; the model becomes obsolete.
- You need to version three things: Code, data, and model—all three.
- Silent failure: A model can fail without crashing, without giving errors, simply by producing incorrect predictions. Catching this requires monitoring.
That's why there is a big difference between a "working model" and a "production-ready model".
Model packaging and presenting
The first step in putting the model into production is packaging it: the model file, necessary libraries, preprocessing code, and version information together as a reproducible whole. Containerization (e.g. Docker: putting the application in an isolated box with all its dependencies) is standard here; It eliminates the "it was working on my machine" problem.
Two basic patterns of serving the model:
- Online/real-time (online): The model sits behind an API, returning an instant prediction for every incoming request. Low latency is critical.
- Batch: The model processes large data sets periodically (e.g. generates scores for all customers at night). Latency is irrelevant, efficiency is important.
Which one is right depends on the business need: instant recommendation online, monthly risk score in batch.
Tip: "Real-time" is a cost, not the default. Batch is much cheaper and simpler if the result will be used within hours. Do you really need an instant answer? Ask that first.
Secure distribution strategies
Opening a new model directly to all traffic is risky; If it's wrong, everyone is affected. Safe distribution patterns:
- Shadow deployment: The new model receives production traffic, but its predictions are not shown to the user, only logged. It is compared with the old model to see if it is safe in real data.
- Canary deployment: The new model is first rolled out to a small percentage of traffic (e.g. 5%); If there is no problem, it is increased gradually.
- A/B testing: Two models are presented to the real user in parallel and business metrics (conversion, clicks) are compared.
- Rollback: Ability to quickly revert to the old version if the new model turns out to be bad. Every deployment should have a rollback plan.
Caution: A deployment without a rollback plan is not complete. Being able to revert to the old version within minutes protects the user when the new model behaves unexpectedly in production. Test this before deployment.
Weak approach / Strong approach
Weak: "The model was good in testing, we went live, we opened it to everyone."
Güçlü: "We containerized the model, labeled it as a version. First, we ran it in shadow mode with production traffic for 3 days, comparing the predictions with the old model — deviation was acceptable. Then we opened it with 5% canary, monitored the throughput metrics and latency. When there were no problems, we gradually increased it to 100%. We had tested the rollback command beforehand."
The difference: the strong approach is gradual, measured and reversible. Risk is limited at every step.
CI/CD and automation
CI/CD (Continuous Integration / Continuous Deployment: pipeline of automatically testing and releasing code changes) in ML covers not only the code but also the data and model steps. A good ML CI/CD pipeline: runs tests when the code changes, performs data validation, retrains the model (if necessary), checks evaluation thresholds, and only advances deployment if the thresholds hold. The principle of “training is automatic, deployment is threshold-based” prevents the bad model from silently leaking into production.
AI is very helpful when setting up these pipelines: writing configuration file (YAML) drafts, test cases, deployment scripts. But you determine the distribution thresholds (whatever metric exceeds what value is published) and rollback policy; these are business risk decisions.
Reproducibility infrastructure
In order to reproduce the behavior of a model in production, model registry: a record that keeps which model was trained with which data and code, and which metrics it received. For each production model, the following should be trackable: training data version, code version (git commit), hyperparameters, evaluation scores, and deployment date. When a problem arises, you should be able to answer the question "which model produced this prediction, with which data?" within minutes. We will deepen this in unit 11.
three mini cases
Case 1 - Problem caught by shadow distribution. A recommendation model beat the old one in testing. Running it with production traffic in shadow mode was found to produce very poor recommendations for a particular segment of users (new users) — the test data was underrepresentative of this segment. The model was fixed without ever being displayed to the user. If it were opened directly, the new user experience would be disrupted.
Case 2 - Irrevocable distribution. A team rolled out a new pricing model to all traffic, with no rollback plans. The model unexpectedly priced some products very cheaply. Reverting to the old version took hours because the process wasn't ready. There was a serious loss of income. Afterwards, mandatory rollback testing was added to every deployment.
Case 3 - Silent data drift. A fraud pattern appeared for months without any errors. But the fraudsters' tactics changed (data drift) and the model's recall silently dropped. Nobody noticed because there was no monitoring. Once a forecast distribution monitoring panel was established, the drift became visible early. We will cover monitoring in unit 8.
Copiable templates
Write a draft deployment plan for this model. Model: [what it does], usage: [online or batch?] Should include:1) Packaging (container, versioning)2) Incremental deployment strategy (shadow/canary/A-B) and why3) Metrics to track (business + technical + latency)4) Rollback plan and how to test5) Deployment thresholds (which metric should exceed what value)
Check this ML CI/CD pipeline:1) Is data validation in the line?2) Can the deployment proceed without holding the evaluation threshold (should it not)?3) Is rollback automatic?4) Are data+code+metrics tracked in the model registry?Pline configuration: [config]
Help me decide whether online or batch presentation is suitable for this model. How long will the result be used: [instant / minute / hour / day]Expected request volume: [number]Is there a delay constraint: [ms]Which one would you recommend in terms of cost and complexity and why?
Write a rollback procedure for this model.- What metric/threshold triggers poor performance?- What are the rollback steps?- How long should the rollback take (target)?- How do I test this procedure before production?
Presentation pattern table
criterion
Online (real time)
Batch
delay
Critical (ms)
insignificant
Usage
Instant response required
Periodic score
Cost
high
low
complexity
high
low
example
Live recommendation, scam
Monthly risk score
Common mistakes
- Distribute without a plan of retrieval. Wrong model hits the entire user.
- Opening directly to 100% traffic. Limit risk with staggered distribution.
- Not establishing monitoring. The model produces errors silently, without error.
- Redundant real-time presentation. While batching is sufficient, cost and complexity swell.
- Not linking model-data-code versions. You cannot reproduce the problem.
- Automatic release with no distribution threshold. The bad model sneaks in silently.
In summary
Moving the model to production is a different and often more difficult engineering task than training it. ML requires extra discipline because it depends on the code-data-model trio: packaging and versioning, delivery pattern (online/batch) that suits the business need, gradual and reversible deployment, threshold-controlled CI/CD and model registration. Artificial intelligence is a powerful aid in generating the code and configuration of this infrastructure; but distribution thresholds, clawback policy, and risk decisions are yours. A distribution without a rollback plan is not complete.
Application task
Containerize (Docker) a model and label it version. Decide whether you will offer online or batch based on your business needs and write your justification. Document a phased deployment plan (shadow or canary) and a tested rollback procedure. Make sure to record the data version, code commit, and evaluation scores in the model registry.
checklist
- [ ] The model is packaged and versioned (container + label).
- [ ] Presentation pattern (online/batch) was chosen according to business need.
- [ ] Staged deployment strategy (shadow/canary) implemented.
- [ ] Rollback procedure written and tested.
- [ ] CI/CD does not advance deployment before the evaluation threshold is met.
- [ ] The model registry holds the data+code+metric link.