The algorithm is not the product

A good algorithm is only a promising component

Medical-software discussions often begin with the algorithm. How accurate is it? How sensitive and specific is it? Does it outperform a clinician, a scoring rule or the previous model? These are important questions—but they do not define the product.

The product is the complete arrangement through which data are acquired, interpreted, presented and acted upon. It includes the intended purpose, users, patient population, data pipeline, interfaces, infrastructure, workflow, instructions, risk controls, support processes and the released configuration operating in the real world.

An algorithm may be impressive in a development notebook and unsafe in clinical use. It may receive different data, be presented at the wrong point in the workflow, produce an ambiguous recommendation, fail silently when a service is unavailable or encourage inappropriate reliance. None of these failures requires the underlying mathematics to be wrong.

The algorithm is therefore best understood as one design element within a medical system—not as a substitute for the system itself.

The fictional ClearPath example

Consider ClearPath, a fictional clinical-decision-support product that estimates a patient’s risk of deterioration during the next 24 hours. Its model performs well against a carefully curated retrospective dataset. The development team can explain its features, report discrimination and calibration, and reproduce the evaluation results.

Once deployed, ClearPath depends on much more. Observations must arrive from the correct patient record, use the expected units and fall within meaningful time windows. Missing values need defined handling. The result must reach the appropriate professional without becoming confused with a diagnosis. Alerts must be noticeable without adding intolerable alarm burden. The system must explain when it cannot calculate a reliable result, and clinicians need a safe route when it is unavailable.

If one of those conditions fails, ClearPath may no longer deliver the medical benefit represented by the original performance study. The model has not necessarily changed; the product context has.

Reliable inputs are part of the medical function

Algorithms do not consume abstract ‘data’. They consume specific fields with assumptions about provenance, units, timing, completeness, preprocessing and clinical meaning. Those assumptions should be treated as product requirements and verified at the interfaces where the software will actually operate.

A blood-pressure value may be entered manually, imported from a monitor or copied from another encounter. A laboratory result may be preliminary, corrected or associated with a specimen collected many hours earlier. A missing value may mean ‘not measured’, ‘not available yet’ or ‘not applicable’. Treating these states as equivalent can alter the output while leaving the algorithm technically functional.

• Define the source, format, units, valid range and acceptable age of every consequential input.
• Specify how identity, encounter and specimen associations are established and checked.
• Control preprocessing, derived variables and reference data as part of the released configuration.
• Detect missing, stale, contradictory and implausible inputs before they become confident-looking outputs.
• Make the effect of input uncertainty visible to users where it affects interpretation.

The user interface changes the clinical effect

The same numerical output can lead to different actions depending on how, when and to whom it is presented. A risk score hidden in a secondary screen may be ignored. The same score delivered as a prominent interruptive alert may be over-weighted. A colour, threshold label or default sorting rule can influence behaviour even when the underlying value is unchanged.

The interface must communicate the intended role of the result, relevant limitations and the action expected from the user. It should also support professional judgement rather than create automation bias. Human-factors work is therefore not decoration applied after algorithm development; it is part of establishing whether the complete product can be used safely and effectively.

Integration and deployment are design concerns

A medical-software product may span a hospital interface, mobile application, cloud service, identity provider, analytics pipeline and external data sources. Each dependency creates assumptions about availability, latency, compatibility, access control and change.

Teams sometimes treat deployment as an operational matter that begins after development. In reality, deployment architecture can determine whether risk controls work. Logging, time synchronisation, rollback, configuration management, monitoring, backup and recovery all influence the ability to deliver and maintain the medical function.

The regulated product boundary also deserves care. A service does not cease to matter because it is hosted by a supplier, and an external component can remain essential to safe performance even when it is not itself classified as a medical device. Contracts and architecture should describe the same operational reality.

Design the degraded state—not only the successful calculation

Demonstrations naturally emphasise the happy path: good data enter, the algorithm runs and a useful result appears. Real products also need deliberate behaviour when data are late, an interface changes, the network is unavailable, a dependency rejects a request or the calculation falls outside its validated conditions.

Failure should be detectable, understandable and proportionate to clinical consequence. A blank field, frozen previous result or generic technical message may be more hazardous than an explicit statement that no current result is available. The safe response might be retry, manual assessment, escalation, restricted operation or suspension of the function. It should not be invented by the user during the failure.

• What failures can occur at each interface and dependency?
• Can the product distinguish no result from a low-risk result?
• Could an old or partial result be mistaken for a current complete result?
• How will the user recognise degraded operation and know what to do next?
• What evidence demonstrates recovery without corrupting records or losing traceability?

Validation must follow the end-to-end workflow

Algorithm evaluation answers whether the model performs appropriately against defined data. Software verification answers whether implemented requirements have been met. Neither alone demonstrates that the product achieves its intended purpose in the hands of intended users and within the intended clinical workflow.

Validation should follow representative journeys from input creation to clinical action, using the production-equivalent configuration and realistic users, environments, interfaces and data conditions. It should explore timing, interruptions, handovers, competing tasks, limitations and foreseeable misuse—not merely confirm that expected screens appear.

This is also where organisational assumptions become visible. Who responds to the result? How quickly? What happens across shift change? Which record is authoritative? Does the user have access to the information needed to interpret the output? A technically correct algorithm cannot compensate for an undefined operating model.

Evidence should describe one controlled product

The development record should connect the intended purpose and clinical claims to system requirements, architecture, risk controls, implementation, verification, validation and release. Model version, parameters, preprocessing, software components, external interfaces and deployment configuration need to form a reproducible baseline.

A common weakness is that each specialist can defend their own component while nobody can defend the complete product. The data scientist explains model performance, the software team explains the application, the cloud team explains uptime and the clinical team explains the intended workflow—but the assumptions between them remain unowned.

Systems engineering closes those gaps. It makes interfaces and operating assumptions explicit, assigns responsibilities and ensures that evidence accumulated by different teams supports the same released configuration.

The practical management question

Leaders should be wary of asking only when the algorithm will be ready. A more useful question is: when will the complete product be ready to perform its intended medical function reliably in its real operating environment?

That reframing changes plans and responsibilities. It draws clinical workflow, integration, usability, cybersecurity, deployment, support and post-market monitoring into development early enough to influence design. It also prevents a successful model demonstration from being mistaken for a nearly finished medical device.

The algorithm matters enormously. It is simply not the product.