Gains:
- Ability to explain the contribution of artificial intelligence to the requirements, architecture and testing stages in medical device and software design.
- Understanding the role of design control and risk management in case the software itself is a medical device (SaMD)
- Ability to understand that AI-supported design outputs must be tested with competent engineer approval, standard and verification tests.
One of the core jobs of the biomedical engineer is medical device design: from an infusion pump to a patient monitor, from a prosthesis to a diagnostic software. Because these devices come into direct contact with the patient, their design differs from ordinary product development; Design controls (disciplined process in which every step from requirement to verification is documented) and risk management are legal obligations. Artificial intelligence contributes to these processes with requirements writing, architectural drafting, test design and documentation. In this unit, we will see where AI fits into device design, how the software itself becomes the device (SaMD), and why AI output is not a substitute for competent engineer approval.
Let's state from the beginning: in safety-critical device engineering, AI is a blueprint and control assistant. If a requirement is missing, a failure mode is missed, a test is out of scope, the responsibility lies with the signing off engineer. AI does not verify the design; The engineer confirms.
Design Chain of Control and the Place of AI
User needs → Design inputs (requirements) → Design outputs → Verification → Validation → Design transfer. This chain is the backbone of device engineering. The role of AI in each ring is different:
- User needs: AI can summarize and theme stakeholder interviews and field notes. Validation: stakeholder confirmation.
- Requirements: AI scans the requirements to see if they are “testable, singular, contradictory” and suggests missing scenarios (edge cases). Validation: engineer review.
- Architecture/design: AI lists alternative architectural approaches and known design patterns. Verification: engineering judgment and calculation.
- Testing: AI generates test case and breakpoint test from requirement. Validation: test coverage matrix.
- Documentation: AI drafts design history file and reports. Verification: technical content checking.
Risk Management: ISO 14971 and FMEA
The standard for risk management in medical devices is ISO 14971; It describes the process of identifying hazards, assessing risk, mitigating it, and justifying the remaining risk. A common tool is FMEA (Failure Mode and Effects Analysis; systematically lists possible failure modes, their effects, and severity/probability/detectability scores). AI is very efficient at brainstorming failure modes for the FMEA chart—reminding modes that the human can skip. But the truth of each line, its score and mitigating measure must be confirmed by the engineer's judgment; A “mitigation” suggested by AI may not actually work or may introduce a new risk.
If the Software Itself is the Device: SaMD
Sometimes the software itself is a medical device: SaMD (Software as a Medical Device; software that works for diagnosis/treatment/monitoring purposes without being embedded in any hardware). An example is an application that produces a risk score from an image or an algorithm that interprets a signal. With SaMD, software cannot be treated as “just software”: design control, risk management, verification/validation, version control and regulatory compliance are mandatory. The IEC 62304 standard defines processes for the software life cycle. A particular challenge in AI-assisted development is that the behavior of the model changes as it is updated; that's why change control and revalidation are critical.
Three Mini Cases: By the Numbers
Case 1 — Requirement gap capture. 140 draft requirements were written for a patient monitor. AI-powered consistency scanning flagged 12 requirements as not testable (such as “must be user-friendly”) and 3 alarm scenarios as missing. The engineering team fixed these; but the two “new requirements” suggested by the AI were actually duplications of existing ones and had to be eliminated. Net gain is through human verification.
Case 2 — FMEA acceleration. In an FMEA study for an infusion pump, the team listed 60 failure modes; AI brainstorming produced 18 additional candidates. Engineers found 9 of them to be genuine and previously omitted, and eliminated 9 as invalid or duplicates. The time savings were real, but filtering was entirely an engineer's job.
Case 3 — Model update risk. A SaMD team updated the underlying model with a “better” version. While the new version has improved overall accuracy, its performance has regressed on a certain device type. Without change control and revalidation this regression would have reached the field. Every model update is a design change and must be verified.
Weak Prompt / Strong Prompt
Weak prompt:
Write the requirements for this device.[idea]
Powerful prompt:
Your role: You are a medical device requirements engineering assistant (YOU ARE NOT THE APPROVAL BODY). Produce a draft requirements for the following device concept: - Keep each requirement unique, testable and verifiable. - Make a separate section for safety/alarm and edge cases. - Mark vague/unmeasurable statements ("easy", "fast") and make them measurable. - At the end, give a list of "open points where the engineer needs to decide". - Standard/clause references as "to be verified" mark, definite indication.Concept:[description]
Four Copiable Templates
1) Requirement quality inspection:
Classify the following requirements as "testable/vague/contradictory/duplicate" and suggest making any ambiguous measurable. List: [requirements]
2) FMEA brainstorming:
List possible failure modes for this subsystem; Suggest effects and possible causes for each. State that the engineer will do the scoring and mitigation. Subsystem: [description]
3) Test scenario generation:
Generate normal, boundary and faulty-input test scenarios for the following requirement; number each scenario traceable to the requirement. Requirement: [text]
4) SaMD change impact analysis:
Write a draft impact analysis checklist for a model release update: affected requirements, revalidation scope, subgroup performance comparison.
Role of the Model: According to Design Phase
Stage
AI contribution
criticality
verification
Need/stakeholder summary
high
low
Stakeholder confirmation
Requirements draft/audit
high
medium
Engineer review
Architecture/calculus
limited
high
Engineering judgment + calculation
FMEA/risk brainstorming
high
high
Engineer scoring/approval
Test scenario generation
high
medium
Coverage matrix
Security approval
None
very high
Authorized engineer signature
Tip: Use AI as a “forgotten scenario reminder” in FMEA and requirements auditing, not as a “decision maker”. Its greatest value is in bringing to the fore the marginal situations that one would miss; But every suggestion must pass through the engineer's filter.
Attention: In SaMD, every model update is a design change. A “better” model may advance in the overall average and regress in a subgroup; No updates should go into the field without change control and revalidation.
Common mistakes
- Accepting the AI recommendation without confirmation. The fitting requirement may produce invalid failure mode or useless mitigation.
- Thinking that SaMD is "just software". Design control, risk management and V&V are mandatory.
- Not verifying model update. Each release is a design change and must be revalidated.
- Passing the vague requirement. Immeasurable statements like "easy/fast" cannot be tested.
- Bypassing engineer approval. The safety decision and signature belong to the authorized engineer; AI is not an approval authority.
In summary
- Medical device design, design control and risk management is a mandatory, documented process.
- AI contributes to the requirements, architecture, FMEA and testing phases with drafts and reminders.
- If the software itself is the device (SaMD), full design control, V&V and regulatory compliance are required.
- Each model update is a design change and requires revalidation.
- AI output is not a substitute for qualified engineer approval; The security decision and signature belong to the engineer.
Application task
Choose a simple medical device concept (for example, a portable SpO2 monitor). Have five requirements drafted using the powerful prompt; followed by asking each “can it be tested?” Check manually and make the vague ones measurable. Finally, write down three failure modes for this device and a mitigation for each, and note which ones you eliminated from what the AI suggested.
checklist
- [ ] I know the chain of design control and the role of AI in each link.
- [ ] I understood the purpose of ISO 14971 risk management and FMEA.
- [ ] I understand the concept of SaMD and its obligations.
- [ ] I understand that the model update is a design change and requires revalidation.
- [ ] I have internalized that the security decision and signature remain with the authorized engineer.