Gains:
- Ability to understand MLOps phases (release, deployment, monitoring, retraining, rollback) and model drift and plan a monitored deployment
- Ability to apply the principles of model fairness, transparency and accountability and distinguish statistical accuracy from ethical acceptability
- Ability to protect personal data with KVKK/GDPR principles and assign final responsibility to a human in high-impact decisions
Training a model and getting a high score is not the end, but the middle. The real value comes when the model is put into production and works reliably, is monitored over time without deterioration, and the entire process is carried out within ethical and legal boundaries. This closing unit combines three topics: MLOps (the discipline of going live, monitoring and maintaining models), ethics (fairness, transparency, non-maleficence) and privacy (protecting personal data). AI produces code, checklists, and blueprints in these areas; But humans decide whether a model goes live, who will be affected, and what data can be used. These decisions are not technical, but responsibility decisions.
MLOps: the model lives like a product
MLOps (Machine Learning Operations - the practice of running, monitoring and updating machine learning models in production) is the adaptation of DevOps in software development to data science. The basic idea: a model is not a file that is trained once and forgotten, but a living product that requires constant maintenance. Major stages:
1. Versioning: Code (Git), data and model are versioned together; It is recorded which model was produced with which data and code.
2. Deployment: The model is put live as an API or batch job. It is usually given to a small audience first (shade/canary distribution).
3. Monitoring: The performance of the model and input data are constantly monitored.
4. Retraining: When performance drops, the model is retrained with updated data.
Pattern shift: silent distortion
The biggest danger in production is model drift (model drift / data drift). The world changes; The conditions on which you trained your model (customer behavior, prices, season, legislation) shift over time and the model begins to become obsolete. For example, a demand model trained before the pandemic is completely wrong during the pandemic. Drift occurs in two forms: data drift (the distribution of input data changes) and concept drift (the relationship between input and target changes). The way to capture these is monitoring: constantly track the input distribution, the prediction distribution, and (if possible) the performance compared to the actual result.
Caution: A model put into production will deteriorate on its own; It's not a matter of "if" but "when." Deploying models without setting up monitoring is like driving a car without ever controlling its engine; One day it just sits there quietly and you don't notice.
MLOps element
Purpose
If neglected
Versioning
Knowing what is produced
Non-reproducible, non-traceable
Monitoring
Seeing the slip early
The model silently breaks down
retraining
stay up to date
Predictions get old
Rollback
return to bad model
Faulty model remains live
Documentation
transparency, turnover
Information gets stuck in one person
Ethics: model decisions affect people
Data models are increasingly used in decisions that affect people's lives: credit, hiring, insurance, justice. With this power comes responsibility. Major ethical risks:
Bias and discrimination: The model can learn and perpetuate injustices in historical data. If a certain group has been given less credit in the past, the model assumes this is a "rule" and automates discrimination. That's why fairness analysis — checking whether the model performs similarly for different groups (gender, age, region) — is essential.
Transparency and explainability: You should be able to explain why a model rejected a person. “The black box said so” is ethically and often legally unacceptable. That's why explainability tools (feature importance, SHAP values) are valuable.
Accountability: Who is responsible if the model makes the wrong decision? The answer is always a person/institution, not a model. Human oversight (human-in-the-loop — a human approving the final decision) should be preserved in high-impact decisions.
Caution: A model may be statistically "correct" but ethically unacceptable. A highly accurate model that systematically disadvantages one group is not a good model. Honesty is no substitute for justice.
Privacy: personal data is carefully protected
The raw material of data science is often personal data, and this data is protected by law: KVKK in Türkiye, GDPR in Europe. The basic principles are: purpose limitation (data is not used for purposes other than the purpose for which it was collected), data minimization (no more data is collected/retained than necessary), anonymization (identifying information is removed) and security (data is kept encrypted and access restricted). Critical rule when working with AI tools: never paste real personal data into a public AI tool. Most of the time, a diagram and an anonymous/synthetic sample are sufficient for analysis.
An additional emphasis in the context of information security: use data science tools and techniques only on data and systems for which you have authority, for defensive and legitimate analysis purposes. Unauthorized access to someone else's data, re-identification of individuals (extracting identity from anonymous data) or unauthorized profiling is both illegal and unethical.
three mini cases
Case 1 — Untracked collapse. An e-commerce company went live with its recommendation model and did not set up tracking. After 4 months the product catalog has changed greatly; the model continued to recommend outdated products, and the conversion rate quietly dropped by 30%. No one noticed for months. Lesson: deployment without monitoring is going blind.
Case 2 — Hidden discrimination. A hiring screening model learned about gender imbalance in historical data and systematically underrated female candidates. It was not noticed because there was no fairness analysis; It was revealed in an audit and the institution faced serious reputational/legal risk. Lesson: group-based fairness control is essential in high-impact models.
Case 3 — Breach of confidentiality. One analyst loaded a file containing real customer emails and purchase history into a public AI tool and said, “summarize segments.” Personal data has left the institution; KVKK process has started. The correct way was to remove identity fields and share only anonymous properties. Lesson: real personal data does not enter the open tool.
Four copyable templates
1) Pre-deployment checklist:
Your role: MLOps consultant. List the things I need to check before putting a model into production: versioning, monitoring metrics, rollback plan, performance threshold, data drift alert, responsible person. Add a one-sentence "why it matters" explanation for each item. I will decide.
2) Model shift tracking design:
Suggest a drift monitoring plan for a classification model in production: (1) what input distributions should I monitor, (2) what alarm for the prediction distribution, (3) how to compare performance when the real result arrives, (4) at what threshold retraining should be triggered. Also provide a code skeleton.
3) Fairness control:
Write code that compares my model's performance for different groups (e.g. age band, region): recall/precision and positive decision rate for each group. Warn if there is a significant difference between groups. Making causal/political interpretations; Just show me the differences and I will consider the decision.
4) Privacy pre-check:
Before giving data to an AI tool, check: is there personal/identity data (name, email, ID, phone, address, IP) in the column list below? If so, list which ones should be removed or anonymized. Columns: [list]. Purpose: to share anonymous schema only.
Weak prompt / Strong prompt
Weak prompt:
My model is ready, go live.
Deployment is not a single step; Going live without monitoring, rollback, fairness and privacy control is a silent invitation for disaster.
Powerful prompt:
Your role: responsible MLOps consultant. My model has been trained, I want full preparation before going live. Generate: (1) pre-deployment checklist, (2) drift monitoring plan, (3) group-based fairness check code, (4) privacy check (is there personal data). Also suggest how to protect human consent in high-impact decisions. The final decisions are mine.
Here distribution, monitoring, fairness and privacy are treated as a single responsible process.
Common mistakes
- Deploying a model without setting up monitoring. The model silently slips and distorts; It can take months to notice.
- Bypassing the justice check. A high-fidelity model may systematically disadvantage a group.
- Making an unexplained black box decision. High-impact decisions must be explainable; "The model said so" is not enough.
- Giving real personal data to open AI tool. KVKK/GDPR violation; The diagram and anonymous example are sufficient.
- Delegating the decision to the model. Responsibility always lies with a person; High impact human surveillance is maintained.
Tip: Before you go live with each model, ask one question out loud: “If this model quietly breaks tomorrow or unfairly penalizes a group, how will I notice and roll it back?” If you do not have a clear answer to this question, the model is not ready for production yet.
In summary
A model's job does not end with a high score, but with working reliably and responsibly in production. MLOps is the discipline of going live, monitoring for drift, retraining, and rolling back the model; Deployment without tracking is silent collapse. Ethics requires fairness, transparency and accountability of the model; statistical accuracy is no substitute for ethical acceptability. Privacy means protecting personal data with KVKK/GDPR principles and keeping real data away from open tools. All these decisions are not technical but responsibility decisions and always belong to a human.
Application task
Think of it as if you were going to put a model you have built (or hypothetical) into production and fill out four lists: (1) pre-deployment checklist, (2) drift metrics you will track, (3) group-based fairness control plan, (4) privacy control. Then write a concrete answer to the question "How will I notice if it silently breaks tomorrow and get it back?"
checklist
- [ ] Am I deploying the model with a monitoring (slip alert) and rollback plan?
- [ ] Have I checked the fairness/performance gap for different groups?
- [ ] Have I maintained human-in-the-loop on high-impact decisions?
- [ ] Have I protected personal data with KVKK/GDPR principles and kept them away from open tools?
- [ ] Have I placed ultimate responsibility on a human, not the model?
Module Exam
1. A data scientist asks the artificial intelligence, "How accurate is random forest on this data?" without training the model at all, and directly puts the answer of "89%" into the presentation. What is the fundamental mistake in this approach?
- A) Waiting for metrics without giving data and models to artificial intelligence; Ignoring that the number it produces is fake and that the real metric can only be found through training and testing ✔
- B) Must use logistic regression instead of random forest
- C) The accuracy rate must always be above 90%.
- D) It is strictly forbidden to include metrics in the presentation
Explanation: Artificial intelligence cannot produce a metric without accessing the model and data; The number he gives is a hallucination (fabricated). The metric is obtained by the data scientist's own calculation only after the model has actually been trained and tested. Unverified output is like an unsigned report.
2. The 'reason for account closure' column is included when collecting data for a churn forecast; This column is filled only after the customer leaves. The model gives 97% on the test set, but does not work in production. What is the name and cause of this condition?
- A) Overlearning; The model is too complex
- B) Data leak; ✔ Using information that will not be available at the time of prediction, but is the result of the target, as a feature
- C) Insufficient learning; The model is too simple
- D) Selection bias; small sample size
Explanation: This is a classic data leak: the 'closing reason' is a result of the target and is still empty at the time of prediction. The model does the trick with this future knowledge, it looks great on the test set but crashes in production because that column is empty. The question 'do I have it at the time of prediction' should be asked to each column.
3. An analyst fills in missing values in the revenue column with the mean; but the missing people actually belong to the low-income segment that has never declared income (systematic missing). Why is this filling incorrect?
- A) The mean is always greater than the median, so it is incorrect
- B) Missing values should never be filled in, they should always be deleted
- C) Filling the systematic gap with the mean distorts the data by artificially representing that group; Filling was done without asking the question 'why is it empty?' ✔
- D) Calculating the average creates a performance problem because it is too slow
Explanation: The cause of the deficiency determines the solution. When the systematic missing (gap concentrated in a certain group) is filled with the average, that group becomes artificially 'average income' and the data is distorted. Before filling it, the question 'why is it empty' should be asked; systematic/significant missing mean should not be filled.
4. A team finds a 0.78 correlation between 'ad spend' and 'sales' and doubles the budget; However, what actually triggers both are seasonal campaigns. What principle in statistics explains this error?
- A) Correlation is not causation; A hidden third variable may be affecting both variables ✔
- B) Since the correlation of 0.78 is too low, the relationship should be ignored
- C) Correlation always proves causation, the team got it right
- D) The correlation between advertising and sales cannot be calculated mathematically.
Explanation: Correlation is not causation. The two variables may act together because a hidden third variable (here seasonal campaigns) affects them both. The conclusion that one causes the other can only be established through experiment and field knowledge; The number of correlations alone is not evidence of causality.
5. A fraud detection model shows 99.2% accuracy and the team celebrates; However, the fake transaction rate in the data is only 0.8% and the model's recall is 6%. What does this table show?
- A) The model is perfect because the accuracy is over 99%
- B) Accuracy is misleading with unbalanced data; The model misses almost all the fakes (low recall), the appropriate metric should be looked at ✔
- C) There is no problem since the recall of the model is high
- D) There was no need to build a model because the fake rate was low
Explanation: 'Accuracy' is misleading in unbalanced data. When the model calls almost every transaction 'clean', it gets high accuracy because fakes are very few, but it fails to catch fakes, which is its main purpose (recall 6%). Therefore, in unbalanced problems, the focus is not on accuracy, but on the confusion matrix and metrics suitable for the purpose of the job, such as recall/precision.
6. In a demand forecasting project, time-dependent data is randomly split into training/testing. The model gives 93% accuracy but crashes in production. What should be the correct division approach?
- A) Enlarging the test set, e.g. splitting 50%/50%
- B) Using a more complex model
- C) Completely remove the partition and train with all data
- D) Chronological splitting: training with the old period and testing with the new period, thus preventing the model from seeing the future ✔
Explanation: If random division is done in time series data, the model sees the future and predicts the past in training; it is a leak and produces success that does not actually exist. The correct approach is chronological splitting: training with the old period and testing with the new period, imitating the real situation in production (predicting from the past to the future).
7. From what data should scaling (StandardScaler) parameters be calculated in feature engineering and how should they be applied?
- A) It should be calculated from all data (training + testing together) so that it is more accurate
- B) It should be calculated separately for each row, from that row's own value
- C) Should be calculated from test data only
- D) It should be calculated only from the training data, then the same parameters should be applied to the test data; otherwise it will leak ✔
Explanation: The parameters of all transformations (mean, standard deviation, etc.) such as scaling, encoding and padding should be learned only from the training data, then the same should be applied to the test data. Taking test data into account also causes test information to interfere with training, that is, leakage, and makes the model look better than it is. Using Pipeline ensures this.
8. A model gives 98% accuracy on the training set and 72% accuracy on the test set. What does this symptom indicate and what should be done?
- A) Insufficient learning; the model should be made more complex
- B) Data leak; test set must be changed
- C) Overfitting; the model should be simplified, regularization and cross-validation should be applied ✔
- D) A normal situation; The education score is always higher anyway, precautions are unnecessary
Explanation: A very high score in training and a significantly low score in testing is a classic symptom of overfitting: the model has memorized the noise of the training data rather than the real pattern. Solutions include simplifying the model, more data, regularization, and verifying the state with cross-validation. Relying solely on the education score hides this pitfall.
9. In a management presentation, the y-axis of a bar chart in which sales increase from 1,000 to 1,020 is started at 980 to make the increase appear huge. The actual increase is 2%. Why is this an ethical issue?
- A) A truncated y-axis visually exaggerates a small difference and misleads the viewer; For fairness, the axis in comparison bars should start from 0 ✔
- B) Bar charts can never be used for sales data
- C) There is no problem; It's always good to make the chart look impressive
- D) Y-axis should always start from the largest value of the data
Explanation: In bar charts for comparative purposes, the y-axis should generally start at 0. Cutting the axis and starting at 980 makes a small 2% difference seem visually huge and misleads the viewer. The ethical responsibility of the data professional is to draw honest graphs that do not overstate or underestimate the data.
10. An analyst finds the total turnover 3 times after JOINing the orders with the product table; The number of lines increased from 240 thousand to 690 thousand. What should have been done to prevent this silent error?
- A) Divide the total turnover by 3
- B) Always use separate queries instead of JOIN
- C) Deleting the product table completely
- D) Checking the number of rows after merge/JOIN and verifying that they are joined via the correct key; Catching replication early ✔
Explanation: When there are multiple rows for each product (different color, for example) in the product table, JOIN via the wrong key will duplicate each order and inflate the totals. The code runs without errors but the result is wrong. The way to avoid this is to check the number of rows after each merge/JOIN and merge with the correct key.
11. A hiring screening model learns about gender imbalance in historical data and systematically under scores female candidates, but its overall accuracy is high. What does this mean?
- A) There is no problem because the model has high accuracy
- B) Statistical accuracy is not a substitute for ethical acceptability; model has learned about past discrimination, group-based fairness check is required ✔
- C) Further increasing the accuracy of the model solves the problem
- D) Fairness is outside the scope of data science
Explanation: Even if a model is statistically correct, it may be ethically unacceptable. The model learns the injustice in past data and automates discrimination. High integrity is no substitute for justice; In high-impact models, fairness control, which measures the performance/decision difference for different groups, is essential and the ultimate responsibility lies with the human.
12. An employee uploads a file containing real customer emails and purchase history into a public AI tool and says 'summarize segments'. Why is this a serious mistake and what is the right way?
- A) There is no problem; AI tools never store data
- B) The error is that the file is too large; should have been reduced
- C) Providing real personal data to an open tool is a violation of KVKK/GDPR; identity fields should have been removed and only anonymous schema/property shared ✔
- D) CSV should have been used instead of just Excel
Explanation: When real personal data (such as e-mail, name, etc.) is given to a publicly available artificial intelligence tool, there will be a privacy violation within the scope of KVKK / GDPR; data leaves the organization. Most of the time, a diagram and an anonymous/synthetic sample are sufficient for analysis. The correct way is to remove identity fields and share only anonymous properties.
13. A recommendation model is put into production but tracking is not established; When the product catalog changes after 4 months, the model continues to recommend old products and the conversion rate silently drops by 30%. What is the name of this phenomenon?
- A) Overlearning; The model memorizes the training set
- B) Model drift; As the world changes, the model becomes obsolete silently, unnoticed because monitoring is not established ✔
- C) Selection bias; sample bias
- D) Data minimization violation
Explanation: This is model drift (model/data drift): when the world changes (catalog, behavior, season) the conditions under which the model was trained shift and the model silently breaks down. A model put into production deteriorates on its own; It's a matter of 'when'. Deployment without monitoring (tracking input and performance) makes it impossible to detect degradation.
14. A model that gets 88% accuracy in a single training/test split has 5-fold cross-validation scores of 88%, 71%, 83%, 64%, 79%. What does this indicate and why is cross-validation important?
- A) The model is stable; 88% is real performance
- B) Cross-validation is unnecessary; One compartment is enough
- C) Large volatility of scores indicates that the model is unstable; cross-validation bases performance on the average and distribution of multiple bins, not a single lucky bin ✔
- D) It is best to report the highest score (88%)
Explanation: A single compartment can be lucky or unlucky; 88% was just the result of that easy division. Cross-validation splits the data multiple times, basing performance on the mean (here ~77%) and showing the volatility of the scores. High volatility here suggests that the model is unstable; Relying on a single compartment is misleading.