Gains:
- Apply evaluation criteria before purchasing an AI tool
- Recognizing the items to look for in the data processing agreement (DPA) and model card
- Create the approved vehicle list and vendor risk scoring
What do you do when a business unit comes to your door and says "we want to use that new AI tool, it will be very useful"? Saying “no way” feeds the shadow AI; Saying "ok" opens up uncontrolled risk. The correct answer is to run a vehicle evaluation process. In this unit we will learn what questions to ask before purchasing/certifying an AI vehicle, what to look for in a data processing agreement (DPA) and model card, and how to distill all of this into an approved vehicle list and vendor risk score.
Why is evaluation necessary?
Every AI tool is a data processor: it processes the organization's data. Approving the wrong tool means handing over the organization's personal data to a third party (and often abroad) in an uncontrolled manner. Core questions to answer before approving a tool:
- Where does it process and store data (which country)?
- Does it use our data in model training? Can it be turned off (opt-out)?
- Does it offer a data processing agreement (DPA)?
- Are there security certifications (e.g. ISO 27001)?
- How long is the chat history kept and can it be deleted?
- Is there a commitment to notify us if there is a security breach?
Data processing agreement (DPA)
Data Processing Agreement (DPA) is a contract signed between the data controller (institution) and the data processor (AI provider) that specifies how the data will be processed. KVKK and GDPR largely mandate this. Items to look for in a DPA:
matter
What should it provide?
Scope and purpose of processing
Let the provider operate only on our instructions
Subprocessors
To whom is it transferred? Is it notified in advance?
Transfer assurance
Standard contract clauses or equivalent
Security measures
Encryption, access control, ISO 27001
Breach notification
Notifying us within a certain period of time in case of a violation
Deletion/return
Commitment to delete/return data at the end of the contract
Right to audit
Ability to audit the provider or receive reports
Beware: Most “free” and “individual” AI plans do not offer a DPA and may use data for model training. For corporate use, corporate/business plans that offer DPA and guarantee education opt-out should be preferred. The free plan is often the plan where data is “paid for”.
Model card and transparency
A model card is a document that explains what an AI model is designed for, what data it is trained with, its limitations and known risks. A good provider shares this. Things to look for in the model card: intended use of the model, known limitations and risks of bias, uses not recommended, and performance/security notes. If the model card is missing or very vague, that in itself is a warning sign.
three mini cases
Case 1 — Cost of the free plan. An accounting team begins processing customer financial data with a free AI tool. The tool does not offer a DPA and states in its terms that it can use the data for model training. The compliance officer notices this and bans the tool, approving a corporate alternative that offers DPA. Difference: a few hundred TL per month for license etc. Possible fines of millions of pounds.
Case 2 — Subprocessor surprise. A company discovers in an audit months later that the AI tool it approved transferred data to subprocessors in three different countries. Since there is no "sub-processors shall be notified in advance" clause in the DPA, the company was not aware of it. Lesson: A clause in the DPA that makes the subprocessor chain visible is a must.
Case 3 — Decision by scoring. An organization establishes an 8-criteria vendor risk score table to compare three AI tools (DPA, data location, training opt-out, ISO 27001, breach notification, deletion, pattern card, price). The rating result highlights the tool that is not the most popular, but the most compatible. The decision is based on a documentable score rather than a subjective “I liked it.”
Tip: Manage the list of approved vehicles as a "whitelist": only allow vehicles that are on the list. The blacklist has to be updated with every new tool and is always one step behind; whitelisting is secure by default.
Exit and addiction risk
The question most agencies skip when approving a vehicle is: “What happens if we want to exit this vehicle?” A good evaluation considers the exit as well as the entry. Two risks stand out. The first is data portability: when you leave the provider, can you get your data and configuration back in a standard format, or is the data locked in the provider? The second is vendor lock-in: business processes may be so tied to a single tool that the exit cost becomes unbearable when the provider increases prices or disrupts service.
That's why it's good practice to also add an "exit plan" line to the confirmation record: how do we get the data back, what is the alternative tool, how long does the transition take. Even if the provider shuts down the service one day, the organization will be prepared.
Caution: Just because a vehicle is popular or cheap does not mean it is sustainable. Small providers may close, be acquired, or suddenly change their policies. Before connecting a critical process to a single tool, consider the exit scenario.
Copiable templates
TEMPLATE 1 — Vendor evaluation question set: "Prepare evaluation questions to ask the vendor before approving a new AI tool. Include: data location, use and opt-out in model training, DPA presence, security certificates, retention period, breach notification, subprocessors, deletion commitment. Include the expected 'secure' answer to each question."
TEMPLATE 2 — DPA clause checklist: "Check the DPA draft below [paste text] for the following clauses: scope of processing, sub-processors, transfer assurance, security measures, breach notification period, deletion/return, right of audit. Mark 'present / missing / uncertain' for each clause. Remind that legal approval is required; do not make final judgments."
TEMPLATE 3 — Vendor risk scoreboard: "Set up an 8-criteria risk scoreboard to compare 3 AI tools: DPA, data location, training opt-out, ISO 27001, breach notification, deletion, pattern card, cost. Let each criterion be 0-3 points, add total and recommendation column. Give blank template, I'll fill it in."
TEMPLATE 4 — Approved tool list entry: "Draft a new entry in the approved AI tool list: tool name, approved intended use, which data classes are allowed (public/internal/confidential), prohibited data types, responsible unit, approval date, review date. In single-line record format."
Weak prompt / Strong prompt
WEAK: “Is this AI tool safe?”-> Repeats marketing promise of model tool; It does not evaluate concrete criteria such as DPA, data location, training usage. STRONG: "I will evaluate this AI tool for enterprise use. What information should I request from the provider based on the following 8 criteria (DPA, data location, training opt-out, ISO 27001, violation notification, storage, subprocessor, model card) and what should be the 'acceptable' threshold in each criterion? Give in checksheet format." -> The model produces a concrete, verifiable evaluation framework.
Common mistakes
- Using free/individual plans with corporate data; Not realizing there is no DPA and opt-out.
- Approving the tool based on marketing promise, not asking for data location and educational usage.
- Sharing data without signing a DPA or checking the subprocessor clause.
- Approving a vehicle with no/uncertain model card without any questions.
- Keeping a ban list instead of a white list and staying behind with every new vehicle.
- Not reviewing the vehicle again after approval (conditions change).
- Basing vendor selection on subjective preference, not a certifiable score.
In summary
- Every AI tool is a data processor; Systematic evaluation is essential before approval.
- The data processing agreement (DPA) is the basic document that binds the data; It should include scope, subprocessor, security, breach and deletion clauses.
- Free/individual plans often do not offer DPA and use data for training; Corporate plans should be preferred.
- The model card shows the limits and risks of the model; Its absence is a warning sign.
- Approved tools should be managed as a whitelist, and vendors should be managed with a documentable risk score.
Application task
Choose three real AI tools your organization might want to use. Set up a vendor risk score table with eight criteria (DPA, data location, training opt-out, security certification, breach notification, retention, model card, cost) and score each vehicle from 0-3 based on these criteria. Then write an approved vehicle list entry for the highest-scoring vehicle: approved intended use, permitted data classes, prohibited data types, responsible entity, and review date. Finally, note five items you'll definitely want to see in a vehicle's DPA and why each is important.
checklist
- [ ] I asked evaluation questions before approving the tool.
- [ ] I clarified the data location and use case in model training.
- [ ] I checked the existence of DPA and its critical items.
- [ ] I examined the model card; I saw the limits and risks.
- [ ] I evaluated the vendor with a documentable risk score.
- [ ] I added the tool to the whitelist with allowed data classes.
- [ ] I set a review date.