Unit 11 / 11

End-to-End Integration, Boundaries and Responsible Execution

Gains:

  • Ability to combine image, signal, data and regulation pieces into a single responsible biomedical AI workflow
  • Ability to apply human-in-the-loop auditing, traceability and post-distribution monitoring principles
  • Ability to maintain the limit at every stage where artificial intelligence does not replace the approval of physicians and engineers

Module Exam

1. Which of the following is the basic principle that determines the risk of an AI output in biomedical engineering?

  • A) The risk of the output is equal to the harm it would cause to the patient if it were incorrect; validation scales accordingly ✔
  • B) AI output is generally safe because it is written fluently and technically
  • C) Validation is unnecessary when the most current model is used
  • D) If the output refers to a manual, no additional checks are required

Explanation: The risk of an output equals the harm to the patient if that output is incorrect. While a minor error in a literature summary is low risk, an error entering an image detection or clinical decision directly threatens patient safety; Therefore, verification intensity scales with potential harm, and every decision involving patient safety is dependent on physician approval.

2. What is the proper role of an AI-supported detection model in clinical use in radiology?

  • A) Automatically approve the radiologist's report and eliminate the workload
  • B) Closing low priority cases without seeing a radiologist
  • C) If the model probability is high, make a diagnosis without additional examination
  • D) Being a support tool that draws attention to the finding and prioritizes it, but the responsibility for diagnosis remains with the radiologist ✔

Description: Image AI is a decision support tool: attracts, prioritizes, measures and flags; however, diagnostic responsibility remains with the radiologist. The model output is not a substitute for the radiologist's decision, it is only an input. The output is used in conjunction with independent clinical evaluation as false positives and false negatives are always possible.

3. What is the most common problem when porting the performance of an AI model from one laboratory to another in digital pathology?

  • A) The model works very fast
  • B) Slide files are too small
  • C) Domain shift caused by dyeing and scanning differences ✔
  • D) Pathologists do not want to use the model at all

Description: Staining protocol, scanner brand, and section thickness vary from laboratory to laboratory; If the model has learned based on the image statistics of one laboratory, its performance will decrease in another laboratory. This phenomenon is called domain shift and is the most common cause of failure in digital pathology; That's why external validation and color normalization are critical.

4. What should be the engineer's first confirmation reflex when an ECG classification model labels 'atrial fibrillation'?

  • A) Writing the label directly into the patient file as a diagnosis
  • B) Check the raw signal, artifacts and physiological plausibility and leave the decision to the physician ✔
  • C) If the confidence score of the model is high, there is no need for any other control.
  • D) Directing the patient to drug treatment

Explanation: The output should first be tested for physiological plausibility and signal quality: baseline drift, muscle artifact or electrode relaxation in the recording may mimic the rhythm. It is wrong to trust the label without looking at the raw signal and R-R intervals; The final rhythm diagnosis belongs to the physician. The model produces a preliminary signal, it does not make a decision.

5. What is the key feature of AI processing of EEG signals that makes verification particularly difficult?

  • A) Low signal-to-noise ratio and high artifact/variability ✔
  • B) The EEG signal is very high amplitude and noiseless
  • C) Recording EEG from only one channel
  • D) EEG data is never digital

Explanation: EEG is a very low amplitude (microvolt range) signal and is easily contaminated by artifacts such as eye blinking, muscle movement, electrode and network noise; Additionally, variability between individuals and recordings is high. This low signal-to-noise ratio causes model outputs to be fragile and necessitates expert neurophysiologist validation.

6. What is the most important limit when working with heart rate data from a consumer wearable device?

  • A) The device's battery runs out quickly
  • B) Storing data in the cloud
  • C) The device is often not medically certified and the signal is susceptible to artifact/bias ✔
  • D) The device is too expensive

Disclosure: Most consumer wearable devices are not certified as medical devices; The PPG (photoplethysmography) signal is affected by motion artifact, and factors such as dark skin tone and tattoos may reduce accuracy. These data are valuable for propensity/awareness, but validated clinical measurement and physician evaluation are required for diagnostic decision.

7. What does 'temporal leakage' mean when building a predictive model with electronic health record (EHR) data?

  • A) Encrypted storage of data
  • B) Leakage of future information into features that does not yet exist at the time of prediction ✔
  • C) The data is too old
  • D) Training the model too slowly

Explanation: Temporal leakage is the interference of training features with information that is not yet available at the time of prediction (future); for example, using discharge diagnosis to predict risk at admission. This leads to high performance in the laboratory but crashing performance in real clinical use and is one of the most insidious flaws of clinical data models.

8. Which of the following becomes mandatory when a software itself is described as 'medical device software' (SaMD)?

  • A) Just working fast is enough
  • B) Does not require any documentation
  • C) Only a beautiful interface is required
  • D) Design control, risk management, verification/validation and regulatory compliance become mandatory ✔

Description: SaMD (Software as a Medical Device) is software that works for diagnosis, treatment or monitoring purposes and does not depend on any hardware. In this case, design controls, risk management (ISO 14971), verification/validation and regulatory compliance are mandatory; The software cannot be treated as 'just software' because its failure directly affects patient safety.

9. What is the most critical risk when you ask AI for an MDR (Medical Device Directive) clause or standard number?

  • A) The model produces fake article/standard references and presents them safely ✔
  • B) The model gives the answer too short
  • C) The model responds in English
  • D) The model responds too slowly

Explanation: Language models can confidently produce nonexistent item numbers, incorrect standard versions, and made-up references (hallucination). In regulatory compliance, an incorrect substance reference leads to serious compliance failure; so each attribution should be verified independently of the official text, AI should be used for navigation/drafting only.

10. What category does health data fall into within the scope of KVKK and what does this mean?

  • A) Ordinary personal data; can be processed freely
  • B) It is not personal data; there is no restriction
  • C) Special categories of personal data; Processed with stricter conditions and protections ✔
  • D) Trade secret only; no privacy required

Explanation: Health data is considered 'special personal data' in KVKK; As a rule, processing depends on explicit consent or exceptions stipulated by law and requires more severe protection measures. Therefore, patient data cannot be entered into uncontrolled tools; Anonymization, access restriction and data processing agreements are mandatory.

11. What does 'calibration' mean in a clinical AI model and why is it important?

  • A) How fast the model runs
  • B) How many parameters does the model have?
  • C) Interface color of the model
  • D) Compatibility of the probabilities given by the model with the actual event frequencies ✔

Explanation: Calibration is the fitting of the probabilities given by the model to the actual frequencies: the event should actually occur in about 20% of the patients for whom it says '20% risk'. Since decisions in the clinic are made based on probability thresholds, a poorly calibrated model will lead to incorrect threshold decisions even if it ranks correctly; therefore, calibration as well as discrimination (AUC) must be verified.

12. Why should 'subgroup performance' be separately checked in image, signal or clinical data models?

  • A) Just total accuracy is always enough
  • B) Because subgroups affect the model speed
  • C) Generic metric can hide underperformance and bias in specific patient groups ✔
  • D) Subgroup analysis is only necessary for marketing

Explanation: While a model may look good on average, it may perform significantly worse in certain subgroups (age, gender, skin tone, device type, hospital); The general metric hides this inequality. In healthcare, this leads to systematic harm in some patient groups; Therefore, performance should be divided into subgroups and reported for fairness.

13. What is the best approach against the risk of 're-identification' when working with patient data?

  • A) Just deleting the name is sufficient under all circumstances.
  • B) Generalize indirect identifiers as well as direct identifiers and restrict access ✔
  • C) If the data belongs to a small number of people, no precautions are required
  • D) Just changing the file name instead of encryption

Description: Even name-deleted data can be re-identified with rare combinations of diagnosis, date of birth, zip code and date. So it's not enough to just extract direct identifiers; Techniques such as date shifting, generalization, k-anonymity and access control should be applied together, and sensitive free text should also be cleaned.

14. What is the primary reason for maintaining end-to-end 'human-in-the-loop' control in the biomedical AI workflow?

  • A) Human control only creates slowness and should be removed
  • B) If the model confidence score is high, human validation is unnecessary
  • C) There is no practical reason other than legal obligation
  • D) AI is subject to bias and error; Patient safety decisions and responsibility should remain with the qualified specialist ✔

Description: Artificial intelligence generates, prioritizes and drafts hypotheses; but the patterns it learns from the data may be incomplete, biased, or out of context. It is imperative that decisions regarding patient safety remain with the approval of the authorized physician and engineer, both to catch errors and to keep the responsibility in the right place; AI does not replace this approval at any stage.