Gains:
- Ability to define quality and performance indicators (SKS, clinical quality, patient safety) and design a meaningful dashboard with artificial intelligence support
- Ability to use AI to produce indicator interpretation, variance explanation and executive summary draft
- Understanding that it is the manager's responsibility to verify the data quality and causality behind the indicator and to recognize the risk of gaming.
"You can't manage what you can't measure" is almost a law in healthcare management. But measurement in healthcare also has a dark side: if you measure the wrong indicator, you reward wrong behavior and undermine patient safety. In this unit, we will learn how to define KPIs (Key Performance Indicators), collect them on a dashboard (dashboard) and use artificial intelligence to interpret these indicators. The limit is clear from the beginning: AI calculates the indicator and produces an interpretation draft; But it is up to the manager to check the accuracy of the data, causality and the risk of gaming.
What would be a good indicator?
The classical framework for measuring quality in health divides indicators into three types. The structure indicator measures resources and infrastructure (such as number of nurses per bed). A process indicator measures how work is done (such as the rate at which antibiotic prophylaxis is given on time). The outcome indicator measures what happens to the patient (such as infection rate, mortality). The name of the quality framework in Türkiye is SKS (Health Quality Standards) and it defines many mandatory indicators. It's important to track these three types together: if you just look at the structure, it says "we have enough resources" but doesn't mean the job is being done well; If you just look at the process, it says "we followed the rules" but it doesn't prove that it is good for the patient. The real picture of quality emerges when all three are read together.
A good KPI has several qualities: it is clearly defined (everyone measures the same thing), measurable, meaningful to the organization, timely, and difficult to manipulate. The biggest danger is the last one. When an indicator is tied to a target, people sometimes find a way to improve the indicator, not the actual performance. This is called target diversion or gaming; It is summarized by the famous Goodhart law: "When a criterion becomes a goal, it ceases to be a good criterion." For example: when the "urgent waiting time" indicator creates pressure, patients can be kept in the "recording open but not seen" status and the indicator can be made artificially good.
In order to read an indicator correctly, it is necessary to look for the answers to three questions together. First, where does the data come from: is the indicator calculated manually or automatically; Is the source reliable? What is the probability of missing/incorrect registration at login? Second, has the definition changed: if “wait time” is measured from registration to examination last month, and from triage to examination this month, the two numbers are not comparable. Third, with which indicator did it move: a number that improves alone may have disrupted the balance indicator next to it. Making a decision by looking at an indicator without asking these three questions is like looking at a single x-ray and treating a patient; Image without context is misleading. Artificial intelligence cannot ask these three questions for you; Because only the manager who knows the organization knows where the data comes from, when the definition changed, and the truth on the ground. The AI's job is to calculate the indicator and list possible explanations; It is up to you to verify which of these statements is true.
Step by step: dashboard and commentary with AI
- Select indicators. Observe the structure/process/result balance; Be sure to include patient safety indicators.
- Clarify definitions. The numerator/denominator, source and period of each indicator should be written.
- Give data anonymously. Just numbers; There is no patient ID.
- Request draft comment from AI. Summarize deviations, trends, and possible explanations.
- Causation and data control. For each "improvement/worsening": has the data quality changed, has the definition changed, is it real or is it gaming?
- Read with associated indicators. An indicator is never evaluated in isolation; It is looked at with pairs (such as LOS–readmission, waiting–gaming).
Caution: AI can mechanically label a sudden change in an indicator as "good" or "bad". Sometimes a sudden drop is not an improvement but a cessation of data entry or a change in definition. It is your job to verify the causality of the comment.
three mini cases
Case 1 — Speeding up the executive summary. Each month, a quality officer manually interpreted 18 indicators and wrote a 4-page summary; This took 3 hours. Anonymous handed the dashboard (just numbers) to the AI and asked for a draft executive summary. AI extracted deviations and possible explanations; the custodian compared each number to his clipboard, corrected two erroneous inferences, and reworked the causal sentences. The duration decreased from 3 hours to 45 minutes; It's not the quality workload, it's the pre-AI preparation that has shortened.
Case 2 — Fake recovery. On one billboard, "urgent average wait time" dropped from 48 minutes to 22 in one month. The AI interpreted this as "significant improvement". The manager became suspicious and investigated the root cause: the registration system had been updated that month, and the "examination start time" was defined differently. So the real hold had not fallen; only the measurement had changed. The AI's fluid interpretation was misleading; The administrator's data control caught the trap.
Case 3 — Catching Gaming. In one hospital, the "outpatient clinic check-up appointment rate within 30 days" was set as a target and quickly increased to 98 percent. The quality team asked AI to cross-read this indicator with associated indicators (check-in rate, patient satisfaction). It turned out that appointments were made, but most of them were not attended; The indicator is good, the actual situation has not changed. The indicator was redefined as "actual control" rather than "given".
Four copyable templates
1) Indicator comment draft:
Your role: assistant to the quality manager.Below is the anonymous monthly dashboard (numbers only):indicator name, this month, last month, target.Task: indicate deviation from target and last month in each indicator, mark the 3 most notable deviations and suggest POSSIBLE explanations.Present explanations as absolute fact; Write it as a hypothesis.
2) Causation/data query:
"Immediate wait time" dropped from 48 to 22 minutes this month. List the possible reasons for this drop under three headings: (1) actual improvement, (2) data/measurement change, (3) target distortion (gaming). Tell me how to check for each one. DON'T DECIDE which one; show control path.
3) Associated indicator pair:
For each main indicator on my dashboard, suggest a "balance/pair indicator" which makes interpreting it alone dangerous (e.g. readmission for LOS). Explain in one sentence why they should be read together. Give it in a table.
4) Board summary:
Draft a 1-page summary for the board from the following anonymous indicator data: 3 good news, 3 areas of attention, 3 action suggestions. Use plain, managerial language. Mark the source of each number as [VERIFY] so I can confirm it.
Weak prompt / Strong prompt
Weak prompt:
Interpret our indicators, tell us whether they are good or bad.
There is no definition, no goal, no context; AI produces mechanical and misleading labels.
Powerful prompt:
Below is the anonymous indicator chart (this month/last month/target). Interpret each deviation relative to the target, but before saying "good/bad", also mention possible data/definition change and gaming possibility. Read the LOS, read the waiting time with the gaming indicator. Do not make final judgments; Mark the points to check.
Indicator type
example
Risk of solitary comment
structure
Nurse/bed ratio
Does not guarantee quality
Process
Rate at time of prophylaxis
Does not show result
Conclusion
infection rate
Volatile in small sample
efficiency
waiting time
Open to Gaming
Common mistakes
- Looking at one indicator. Each major indicator should be read with a balance indicator.
- Immediately considering sudden change as good/bad. There may be data or definition changes.
- Ignoring Gaming. The target-dependent indicator can be manipulated.
- Mistaking the AI interpretation for causality. AI sees the relationship, not the cause.
- Not questioning data quality. Good indicators cannot come from bad data.
Tip: Add a small “balance indicator” to your dashboard next to each main indicator and always ask for the AI commentary with this pair. A number that shines alone often hides its dim pair.
In summary
Quality indicators measure the health of the organization, but wrong measurement breeds wrong behavior. AI is a powerful aid in calculating indicators, summarizing variances and producing a draft management summary. But AI sees relationships, it cannot establish causality; Sudden change may be data/definition change or gaming. Read each indicator with balance pair, query data quality, verify causality. The final approval of the comment and decision lies with the administrator.
Application task
Get 5-6 anonymous indicators (this month/last month/target) from your organization. Request comments from the AI with the “Indicator comment draft” template. Select the indicator that changes the most and extract three possible explanations (fact/data/gaming) with the "Causation/data query" template and write down how you would check each in 5 points.
checklist
- [ ] Have I clarified the definition of indicators (numerator/denominator/source)?
- [ ] Did I provide data anonymously?
- [ ] Do I read each major indicator with the balance pair?
- [ ] Have I checked the possibility of data/definition/gaming in case of sudden changes?
- [ ] Did I, as the administrator, approve the final comment and decision?