End-to-End Workflow, Quality Management and Self-Audit — Artificial Intelligence in Radiology and Medical Imaging

Gains:

Ability to design an end-to-end workflow that shows where and with what approval artificial intelligence comes into play in the entire chain, from pre-shooting to report signature.
Ability to establish performance monitoring, feedback loop and regular self-audit in artificial intelligence-supported processes
Ability to make the use of artificial intelligence in an imaging unit sustainable with the principles of governance, responsibility and continuous improvement

In the previous ten units, we have covered the pieces of AI in radiology: triage, structured reporting, measurement automation, PACS integration, metrics, modality-specific tools, image quality, tracking, and privacy. In this final unit, we will combine the pieces and establish an end-to-end workflow: a holistic map that shows where and with what approval artificial intelligence comes into play in the chain that an audit passes from the moment of request to the signed report. Then we will consider the two things that make this flow sustainable: quality management (monitoring performance, feedback loop) and self-regulation (regular review, governance).

Core principle: Safe use of AI-powered imaging is not a one-time installation; It is a living process that requires constant monitoring of performance, a feedback loop, review of error patterns, and regular self-audit. At every step, responsibility, signature and final decision remain with the radiologist.

End-to-end flow: Where is AI, which confirm

Let's follow the journey of an audit, including the points where AI comes into play and human approval at each point.

Request and indication: The clinician requests an examination. AI can recommend indication suitability and protocol selection; The technician/radiologist approves. (Unit 8)
Shooting and quality: Modality produces the image. AI dose optimization, denoising and artifact flagging; The technician decides on quality and reshoots if necessary. (Unit 8)
Triage: The image falls into the PACS. AI prioritizes time-critical events; The radiologist still reads each exam. (Unit 2)
Reading and detection: The radiologist reads. CAD marks as second eye; The radiologist evaluates what is marked and what is not. (Unit 6, 7)
Measurement and follow-up: AI measures and compares with previous examination; The radiologist verifies consistency. (Unit 4, 9)
Reporting: AI produces structured draft; The radiologist matches each sentence with the image and signs it. (Unit 3)
Critical finding notification: AI reminds of critical finding; The radiologist confirms, communicates to the clinician, and completes the closed-loop. (Unit 2, 6)
Archive and trace recording: AI output is stored as a separate layer; The original is not changed, the trace record is kept. (Unit 5)

Stage

AI role

human approval

Connected unit

Request/protocol

Suggestion

Technician + radiologist

Shot/quality

Dose, denoising, artifact

technician

triage

priority flag

Radiologist (still reads)

Read/detect

CAD second eye

Radiologist (diagnostic)

6, 7

Measurement/tracking

Calculation, comparison

Radiologist (corrects)

4, 9

Reporting

draft

Radiologist (signature)

critical finding

reminder

Radiologist (transmits, confirms)

2, 6

Critical finding reporting: closed-loop

A critical finding (major bleeding, massive embolism, tense pneumothorax) is not considered communicated simply by writing it in the report. Closed-loop communication means confirming that information reaches and is received directly by the attending physician. AI can recall a critical finding or even produce a draft notification; However, it is the radiologist's responsibility to confirm the finding, communicate it to the correct physician, document the communication, and close the loop. Automated alerting is not a substitute for human communication.

Attention: Just because an AI critical finding alert appears on the worklist does not mean that the notification has been made. Notification is completed only when the radiologist confirms the finding, reaches the attending physician, and receives the information. It is human's job to close the closed-loop.

Quality management and self-audit

An AI-powered process can break down over time: the model is updated, the device changes, the population shifts, the false positive rate increases. Therefore, performance must be constantly monitored.

Performance monitoring: Collect false positive/negative samples; Track triage flag rates and true positive percentage.
Feedback loop: Systematically collect radiologists' "this flag was wrong/missed this" feedback and share it with the manufacturer.
Error review: Review missed findings and false alarms at regular meetings; Distinguish the root cause (model, protocol, human).
Distribution shift tracking: Repeat local verification when device/protocol/population changes.
Governance: Who uses which model, with which version, in what context; Who is responsible - these should be in writing.

three mini cases

Case 1 — The feedback loop works. One unit sees from monitoring data that the triage model's pneumothorax false positive rate has increased to 75% within months. Radiologists collect their feedback, review the threshold with the manufacturer; the rate drops to 45% and alarm fatigue decreases. Without monitoring, the model would quietly become unreliable.

Case 2 — Missed finding root cause analysis. Two missed small nodules are noted at the one-month review. The analysis shows that both occur on portable graphs and on a device on which the model was not trained (distribution drift). The unit narrows the scope of the model on portable radiographs and alerts radiologists. The mistake was used to fix the system rather than blaming one person.

Case 3 — Critical finding is closed-loop. AI warns of major bleeding on a brain CT. The radiologist confirms in 90 seconds, calls the neurosurgeon, verifies receipt of information, and documents communication (time, person, content). The closed-loop is completed. AI reminded; The radiologist made the notification and confirmation. If it were left to automatic alerting, there would be no assurance that the physician received the information.

Weak prompt / Strong prompt

Weak prompt:

Set up AI workflow in radiology, automate everything.

“Automate everything” is dangerous in a security-critical domain; Ignores human checkpoints and safe-default.

Powerful prompt:

Your role: ASSOCIATE consultant to the radiology unit on end-to-end AI workflow design.Decision making; produce outline and checklist. Specify where the AI is added in the request→capture→triage→reading→measurement→reporting→critical finding→archive chain and indicate human approval at EVERY step. Also suggest metrics for performance monitoring, feedback loop, and self-auditing. Principle: as risk increases, AI role becomes smaller, human approval grows; diagnosis/signature/notification at the radiologist. Unit context: [write]

The strong will centers on human approval points, monitoring, and self-control.

Copiable prompt templates

END-TO-END FLOW DRAFT TEMPLATEYour role: ASSISTANT consultant. For the request→imaging→triage→reading→measurement→reporting→critical finding→archive chain, produce a table that lists the role of AI and human approval at each stage. Mark the risk level and leave no safety-critical decisions "taken over" by the AI. Unit context: [write]

PERFORMANCE MONITORING TEMPLATEI will give you a period of triage/CAD data. Calculate false positive rate, number of missed findings, and percent true positive; warn of alarm fatigue and distribution drift; Recommend at what threshold the manufacturer/model should be reviewed. The decision is in the institution. Data: [write]

ROOT CAUSE REVIEW TEMPLATEI'll give you a missed find/false alarm case. Distinguish root cause: model (training/scope), protocol/device (distribution drift), human (automation bias/alarm fatigue). Produce suggestions focused on system correction, not blame. Event: [write]

CRITICAL FINDING CLOSED-LOOP TEMPLATEProduce a closed-loop checklist for a critical finding notification: confirmation of finding, identification of correct physician, direct transmission, confirmation of receipt of information, documentation of communication (time/person/content). Remind us that automatic alerts do not replace notifications. Context: [write]

Common mistakes

"Automating everything". Security-critical decisions require human approval; Full automation is dangerous.
Installing it once and not watching it. The pattern breaks down over time; Continuous performance monitoring is a must.
Using the mistake to blame the person. Root cause analysis should focus on system correction.
Leave the critical finding to automatic warning. Closed-loop is closed by the radiologist; Transmission and confirmation is human work.
Not putting governance in writing. Who, which model, which version, which scope, who is responsible — must be documented.

Tip: Ask this question regularly in your unit: “If we turned off AI today, would we continue to operate safely?” The answer should always be "yes". AI speeds up reading, but it will never replace reading, recognition and signing.

In summary

Artificial intelligence comes into play at many points in the chain from the moment of order to the signed report in radiology: protocol recommendation, quality, triage, detection, measurement, follow-up, reporting and critical finding reminder. The holistic principle is the summary of the entire module: as risk rises, the role of AI shrinks, human consent grows; At no step does the AI take over the reading, diagnosis, signature, or critical finding reporting. What makes this flow sustainable is recognizing that safe use is a living process, not a one-time setup: performance monitoring, feedback loop, bug review, distribution drift tracking, and written governance. When reporting a critical finding, the radiologist closes the closed loop. Underneath everything stands the unchanging sentence: the decision belongs to the person.

Application task

Draw an end-to-end AI workflow chart for your unit (or a sample organization): write down the AI's role, human approval, and risk level at each stage. Then determine the three metrics (false positive rate, missed findings, percent true positive) and action thresholds that you will track with the “Performance Tracking” template. Select a critical finding scenario and write the steps using the "Critical Finding Closed-Loop" template. Finally, "Would we work safely if we turned off AI today?" Answer the question honestly for your unit and note the deficiencies.

checklist

[ ] I defined human validation of each AI point in the end-to-end flow.
[ ] I have not delegated any safety-critical decisions (diagnosis/signature/notification) to the AI.
[ ] I determined performance monitoring criteria and action thresholds.
[ ] I set up the feedback loop and bug review process.
[ ] I made a plan to repeat local verification for distribution shift.
[ ] I ensured that the radiologist closed the closed-loop on critical finding notification.
[ ] "Will we work safely if we turn off AI?" I can say yes to the question.

Module Exam

1. A radiology triage AI marked a brain CT as 'low priority' and moved it to the end of the worklist. What should the radiologist do for this examination?

A) Still reads and reports the audit completely; triage only changes the order, not eliminates reading ✔
B) Artificial intelligence normally closes the study without reading it because it says it is low priority.
C) Automatically postpones the examination to the next day
D) It only looks at the area marked by artificial intelligence and cuts the report short

Description: Triage AI only changes the reading order; It cannot exclude any examination from reading. 'Low priority' is not a diagnosis and may be false negative. Each examination must still be fully read and reported by the radiologist; It does not replace reading the priority signal.

2. What combination of risks are the AI missing a real pathology (e.g. a small pneumothorax) and the radiologist relying on it and not adequately examining the image?

A) False positive and alarm fatigue only
B) False negative and automation bias (over-reliance on AI) ✔
C) Only poor image quality
D) DICOM header error only

Explanation: It is a false negative if the model misses the finding; Automation bias is when the radiologist overly trusts artificial intelligence and relaxes his own examination. When the two combine, the raison d'être of human monitoring disappears and the finding can be missed entirely. Therefore, areas not marked by artificial intelligence should also be read.

3. AI added a sentence 'right adrenal adenoma' in a draft of a thorax CT report that had no equivalent in the image. What should the radiologist do?

A) Considers the sentence reliable and signs it directly.
B) Since he is not sure, he rewrites the entire report to the artificial intelligence and signs it.
C) Matches the sentence with the image; If there is no provision, he removes it and signs the draft as verified ✔
D) Leaves the sentence but writes 'artificial intelligence added' at the end

Explanation: Language models can fluently fit a finding not present in the image; this is called hallucination. The radiologist must match each sentence in the report to the image, remove any irrelevant phrases, and never sign the draft without verifying it. The draft report is a start, not proof.

4. A specialist uploads a DICOM image with the patient's name and protocol number without anonymization into a publicly available AI tool. What is the main problem here?

A) Image resolution is insufficient for artificial intelligence
B) Artificial intelligence cannot read DICOM format
C) No comment can be made because the image is black and white
D) Identity data in the DICOM header and embedded in the image is shared without anonymization; This is a KVKK violation ✔

Description: DICOM files carry both information such as patient name, identity, date of birth in the header, and sometimes identity data embedded in the image (burned-in). Uploading these to an external tool without anonymizing them is a disclosure of sensitive health data and a violation of KVKK. The header and embedded data must be cleared first.

5. What is the most important verification step when comparing an automated lung nodule measurement (e.g. 8 mm) produced by AI to a manual measurement (5 mm) from a previous exam?

A) Verify that the measurements are compared with the same method, plane and segmentation consistency and evaluate the discrepancy by the radiologist ✔
B) Assuming the new measurement is always accurate and reporting growth
C) Taking the average of two measurements and reporting them
D) Ignore the measurement completely and write only visual impressions

Explanation: The measurement is an estimate and depends on the segmentation boundary, slice selection, protocol and measurement method. When automatic and manual measurement are done using different methods, a false impression of 'growth' may arise. Before making an enlargement decision, a comparison should be made using the same method, on the same plane and with consistent boundaries, and any inconsistency should be evaluated by the radiologist.

6. What is the golden rule regarding the original DICOM image when adding AI results to PACS?

A) Artificial intelligence markings must be permanently written over the original image
B) AI output is added as a separate series/layer; the original diagnostic image is not changed and the trace recording is preserved ✔
C) The original image should not be stored at all, only the processed version should be kept
D) AI output should be embedded as text within the patient header

Description: AI markups, metrics, and highlights should be stored as a separate series, layer, or structured report object; The original diagnostic image should never be altered. This way, the radiologist can always see the raw data, the trace recording is preserved, and the AI output remains distinguishable from the original.

7. A mammography AI works with high 'sensitivity' but low 'specificity'. What does this mean in practice?

A) The model marks almost no findings
B) The model never produces false positives
C) The model captures most true findings but can produce many false positives, leading to unnecessary recall ✔
D) The model only marks cases for which it can make a definitive diagnosis.

Explanation: High sensitivity means capturing most true positives (few false negatives). Low specificity means marking healthy cases frequently, which means producing a large number of false positives. This reduces leakage but increases the burden of unnecessary recall and investigation; The choice of threshold is a clinical balance and is governed by the radiologist's interpretation.

8. An AI model is trained on a specific manufacturer's device and a specific patient population. What is the best approach before using it on a different device and population in your own institution?

A) Assuming that the model will give the same performance everywhere
B) Mandatory all inspections without ever testing the model
C) Considering it safe only if the user interface is in Turkish.
D) Perform local validation and check performance on your own device and population and review the level of approval/evidence ✔

Explanation: Performance may decrease when moving away from the data distribution on which the model was trained; This is called distribution shift. Prior to use in a new device, protocol, or population, local validation (performance check on your own data) should be performed and regulatory approval and level of evidence should be reviewed.

9. What is the most important security advantage of structured reporting over free text reporting?

A) Thanks to standard headings and mandatory fields, it reduces the risk of missing findings and ambiguity and ensures consistency ✔
B) Makes reports appear shorter
C) It makes it unnecessary for the radiologist to look at the image.
D) Allows artificial intelligence to send the report without radiologist approval

Description: Structured reporting uses standard headings, required fields, and a common glossary; This reduces the risk of missing findings, missed fields, and misunderstandings, ensures consistency among clinicians, and makes it easier for artificial intelligence to fill in fields and check consistency. Free text is more flexible but carries more risk of omission and ambiguity.

10. An image enhancement (super-resolution) algorithm can 'clean up' a low-dose CT while adding fine structure that is not actually present. What does this remind you of?

A) Improved image is always more reliable
B) Enhancement may add artificial detail or delete actual finding; In case of doubt, the raw image should be returned and diagnostic validity should be questioned ✔
C) After denoising, the dose decreases automatically
D) Enhancement only changes color, does not affect the content at all

Explanation: While image enhancement algorithms reduce noise, they can add artificial detail or erase real small findings. This is a reminder that the diagnostic validity of the enhanced image may be limited and in case of doubt, return to the raw/original image and, if necessary, re-evaluation with the appropriate protocol is necessary. Looking good aesthetically does not mean diagnostic accuracy.

11. What is the correct behavior when anonymizing a patient story to be sent to a public AI tool?

A) It is sufficient to delete only the surname and leave the name and protocol number
B) Deleting all clinical information as well so the AI sees no context
C) Remove identity, protocol, contact and institutional information and leave only clinically necessary anonymous information ✔
D) Anonymization is unnecessary; the vehicle is already safe

Explanation: Name, TR ID number, protocol/file number, contact and institution information should be removed; Only clinically necessary information (age group, gender, relevant history) should be left. The correct approach is to write '62-year-old woman, known history of breast cancer' instead of 'Ayşe Yılmaz, protocol 2024-114523'.

12. The AI flagged no findings on a pulmonary embolism (PE) CT angiogram (negative AI). What is the correct interpretation of this for the radiologist?

A) There is definitely no embolism, the report can be quickly closed as negative
B) The examination should be repeated
C) There is no need to consult a clinician because the artificial intelligence says negative
D) Negative AI is not a guarantee; The radiologist reads the entire image with his own assessment, the model may have missed the finding ✔

Explanation: A negative AI output does not mean that embolism is excluded; the model may have missed the finding (false negative). The radiologist must read the entire image with his or her own clinical judgment, including areas not marked by the AI. Negative AI is not a guarantee of diagnosis, but at most an auxiliary signal.

13. The AI generated an alert for a critical finding (e.g., massive intracranial hemorrhage). Who is responsible for closed-loop critical finding reporting?

A) It is the radiologist's responsibility to confirm the finding, communicate it to the clinician, document the communication, and complete the closed-loop ✔
B) Notification is completely automatic, the radiologist has no role
C) Responsibility passes to the artificial intelligence producer
D) Since the warning is in the worklist, no additional communication is required

Description: The AI warning is a reminder; It is the radiologist's responsibility to confirm the critical finding, communicate it directly to the attending physician, document the communication, and complete the closed-loop (confirmation of receipt of information). Automated alerting is not a substitute for human communication and confirmation.

14. What is the best governance approach to ensure the use of artificial intelligence in an imaging unit is sustainable and safe?

A) Install once and never review again
B) Monitor performance regularly, establish a feedback and self-audit cycle, review examples of errors, and keep the final decision with the radiologist ✔
C) Delegating all decisions to the model and reducing the number of radiologists
D) Never recording artificial intelligence outputs and leaving no trace

Description: Safe use is not a one-time installation; It requires regular monitoring of performance, collection of radiologist feedback, review of false negative/positive samples, monitoring of model updates and distribution drift, and regular self-audit. Responsibility, signature and final decision always remain with the radiologist.