AI has accelerated medical-device development—but not manufacturer responsibility

Ideas prompted by Steven Dodsworth and Marcelo Duhalde

This article was prompted by Steven Dodsworth’s LinkedIn article, ‘AI Has Changed Everything and Nothing: Strategy, AI & Digital Health’, published on 29 August 2026. His argument that AI changes opportunity and complexity without removing the fundamentals of strategy provides the starting point.

It also draws on a timely LinkedIn post by Marcelo Duhalde, who asks whether organisations are automating and ‘hiring’ agents too early. Referencing Peter Drucker and David Heinemeier Hansson, he argues that our new capabilities should be applied with focus and discipline to the challenges we must tackle—not merely those we can. The discussion below extends both contributions into medical-device development, evidence and manufacturer accountability.

AI changes the speed of work

Medical-device teams can now use generative AI to explore concepts, structure requirements, identify hazards, draft software, create test cases, summarise evidence and prepare documents in a fraction of the time these activities once required.

Used intelligently, AI can help experienced people consider more alternatives, find inconsistencies earlier and spend less time on routine drafting. It can also produce an impressive amount of material before the organisation has decided what it is building, who will use it, which risks matter or what evidence will be needed.

Faster output is not automatically faster development. If AI helps a team reach the wrong conclusion sooner—or creates documents that disguise unresolved thinking—the apparent productivity becomes future rework.

The manufacturer still owns every consequential decision

AI does not become the legal manufacturer. It does not approve the intended purpose, accept residual risk, authorise design release or defend the resulting evidence before an auditor, notified body or regulator.

An AI-generated answer may contribute to that work. It cannot carry the accountability for it. Activity can be delegated; accountability cannot. AI simply makes the boundary less visible because the output appears immediately and may arrive without an identifiable expert standing behind it.

• What problem is the device intended to solve?
• Which patients, users and environments are covered?
• Why are the selected architecture and design appropriate?
• How were hazards, hazardous situations and foreseeable misuse identified?
• What was verified and validated, using which configuration and acceptance criteria?
• Who reviewed and approved each important conclusion?

A convincing document is not objective evidence

Generative AI is particularly good at producing plausible structure and fluent prose. In a regulated development environment, that strength can become a weakness.

Document quality is not measured by how professional the document looks. It is measured by whether its claims are correct, its inputs are controlled, its reasoning is reviewable and its conclusions are supported by objective evidence.

• Important omissions hidden by otherwise comprehensive coverage.
• Inconsistent assumptions across separately generated documents.
• Generic controls presented as though they address device-specific risks.
• Confident conclusions that exceed the available clinical or engineering evidence.
• Traceability created after the event rather than maintained through decisions.
• Outputs whose origin, model version, prompts or human review cannot be reconstructed.

Start with the problem—not the availability of AI

AI capability is increasingly expected, but it is rarely a sufficient product strategy. Adding a model to a weak value proposition does not strengthen it. It often adds new development, evidence, usability, data-governance and lifecycle obligations.

The question is not, ‘Where can we add AI?’ It is, ‘Does this capability improve the intended medical outcome sufficiently to justify the new risks and controls it introduces?’

• Who has the unmet need?
• What decision or action will the product support?
• Who buys it, who uses it and who bears the consequences of failure?
• How will it fit into the clinical workflow?
• What benefit is claimed, and what evidence can realistically support that claim?
• What happens when the output is uncertain, unavailable or wrong?

Distinguish three materially different uses

Governance should begin by defining the actual role the system performs. Calling every application ‘AI’ obscures materially different risks.

• AI as a development aid: drafting, coding assistance, document review, test generation and data exploration require controls for competence, verification, confidentiality, provenance and record integrity.
• AI as part of the medical-device function: prediction, classification, interpretation or patient-specific guidance makes the model part of the product architecture and evidence argument.
• AI in an operational process: complaint triage, vigilance support, production inspection or regulatory intelligence must be controlled according to the consequence of a wrong or missed output.

Probabilistic systems require deliberate evidence

Traditional software is often expected to produce the same result from the same inputs. AI-enabled functions may instead produce outputs whose performance is described statistically across populations and conditions.

An overall accuracy figure is rarely enough. Teams may need to understand sensitivity, specificity, false-positive and false-negative behaviour, confidence calibration, subgroup performance and the consequences of distribution shift. Statistical performance, data quality and uncertainty are design inputs—not merely post-development analysis.

• Does the evaluation dataset represent the intended population and use environment?
• Are training, tuning and test data appropriately separated?
• Are clinically important subgroups visible rather than hidden within an average?
• Can users recognise when they should not rely on an output?
• How will performance be monitored after deployment?
• Which changes require new verification, validation or regulatory assessment?

AI can increase the need for design control

When concepts, code and documents can change quickly, the organisation needs stronger control over baselines, configuration, review and decision authority. Otherwise, teams may be unable to establish which model, data, prompt, software configuration or requirement set produced a particular result.

This need not become an enormous new bureaucracy. A short, risk-based policy combined with practical review criteria can be more effective than either unrestricted use or a nominal prohibition that employees quietly work around.

• Which AI tools are approved for which activities?
• What information may or may not be entered into them?
• When must their use be recorded?
• What level of human competence and review is required?
• How are generated requirements, code and tests verified independently?
• Who can accept the result and on what evidence?

AI literacy must be cross-functional

AI governance cannot sit solely with software engineering or information technology. Product leaders, clinical specialists, quality and regulatory teams, cybersecurity and privacy specialists, and human-factors professionals each see different parts of the risk and evidence problem.

The most dangerous position is neither enthusiasm nor scepticism. It is superficial confidence: an organisation using AI extensively while assuming that one technical specialist—or the tool itself—understands all the consequences.

Avoid pilotitis

AI makes demonstrations easy. A team can produce an appealing prototype quickly, show promising examples and create momentum before the underlying product problem has been resolved. But the pilot is not the medical device.

A deployable product also needs controlled data, robust architecture, cybersecurity, usability, validated performance, integration, monitoring, change management, support arrangements and a viable adoption model.

• Which uncertainty is the pilot intended to reduce?
• What evidence would justify moving to the next stage?
• What evidence would cause us to stop?
• What must be true for the capability to operate safely at scale?
• Who will own the system after the demonstration team moves on?

The practical conclusion

AI has changed the economics and speed of knowledge work. It has not changed the need to define the right product, understand its risks, produce objective evidence or assign accountable people to consequential decisions.

The organisations that benefit most will combine AI’s speed with experienced judgement, disciplined design control and the willingness to challenge a convincing answer before relying on it.

The key question is therefore not, ‘Did AI produce this?’ It is, ‘Do we understand it, can we verify it, and are we prepared to accept responsibility for using it?’