AI! AI! Oh?

AI! AI! Oh?

AI! AI! Oh? Where will it go?Will it help us to flourish, or steal the whole show?It answers our questions, creates at command—Yet sometimes invents what it doesn’t understand.

The extraordinarily capable assistant

Artificial intelligence has become difficult to avoid. It writes, summarises, translates, analyses, illustrates and advises. It can produce in seconds what might once have required hours of work—and it does so with an apparently effortless confidence.

That confidence is part of its appeal. It is also part of the danger.

Used well, generative AI can be a remarkable working partner. It can help us confront the intimidating blank page, organise unstructured thoughts, compare possible approaches and identify questions that we may have overlooked.

For engineers, it can explain unfamiliar technology, suggest test cases, review code and help structure technical documentation. For quality and regulatory professionals, it can summarise requirements, identify potential gaps and improve the clarity of procedures or reports. For managers, it can turn fragmented notes into coherent plans, presentations and communications.

None of these activities necessarily removes responsibility from the human user. At its best, AI increases the reach of an experienced person. It makes knowledge easier to access and routine intellectual work faster to complete.

This is not simply automation. Traditional automation follows rules established in advance. Generative AI can work with ambiguity, language and incomplete ideas. It can help us think—not because it thinks exactly as we do, but because interacting with it encourages us to articulate, question and refine our own reasoning.

That is tremendously valuable.

The machine that does not know what it knows

The difficulty is that an AI-generated answer may look equally convincing whether it is correct, partially correct or entirely invented.

AI does not experience uncertainty in the way a competent professional does. A human expert may pause, recognise the limits of their knowledge and say, ‘I need to check that.’ An AI system can instead continue producing plausible prose.

The result may be a fabricated reference, an incorrect interpretation of a standard, an outdated regulatory expectation or a technical recommendation that works in principle but fails in the particular circumstances.

Fluency is not evidence.

This matters in any field, but it matters especially where decisions affect safety, health, privacy or legal compliance. A beautifully written answer is not necessarily a defensible answer. A complete-looking document is not necessarily a compliant document. A plausible risk analysis is not necessarily based on the actual device, its architecture or its intended use.

AI can help prepare the work. It cannot silently inherit accountability for it.

AI in medical technology

The medical-device sector has good reason to be interested in AI. Developing regulated products involves an enormous volume of information: user needs, requirements, risks, architecture, verification evidence, clinical data, cybersecurity records and post-market information.

AI could help us navigate this complexity. It may assist with traceability, detect inconsistencies, identify missing information and make specialist knowledge available to smaller organisations. It could help engineers understand regulatory expectations and help regulatory specialists understand technical design decisions.

But medical-device development is not merely a documentation exercise.

A system can produce a polished risk table without understanding the physical sequence of events that could harm a patient. It can generate software requirements without knowing whether they accurately represent the product. It can propose verification tests without recognising the unusual boundary conditions that matter most.

If we use AI only to generate more documents more quickly, we may simply create better-presented confusion.

The real opportunity is to use it to improve the quality of our thinking: to challenge assumptions, expose gaps, explore alternative failure scenarios and connect information that would otherwise remain trapped in separate documents or departments.

Assistant, reviewer or decision-maker?

The right role for AI depends on the task and its consequences.

For low-risk work, such as improving wording or generating an initial list of ideas, light review may be sufficient. When AI is used to interpret a regulatory requirement, propose a safety control or influence a design decision, much stronger verification is needed.

A useful principle is: the greater the consequence of an error, the stronger the human review and supporting evidence must be.

That review should not be ceremonial. Asking someone to approve a long AI-generated document without providing the source material, assumptions and reasoning merely transfers the burden downstream.

We need to know:

• What information was provided to the AI?
• What information might have been missing?
• Which statements are factual, and how were they verified?
• Where did professional judgement enter the process?
• Who is accountable for the resulting decision?
• Could confidential or personal information have been exposed?
• Can the result be reproduced or explained if it is later challenged?

Learning to work with AI

These are not arguments against using AI. They are the conditions for using it responsibly.

The most important AI skill may not be prompt engineering. It may be judgement.

People need to learn when to trust, when to verify and when not to use an AI-generated result at all. They must recognise the difference between asking AI to improve an explanation and asking it to make a safety-critical determination.

Organisations also need proportionate governance. Blanket prohibition is unlikely to succeed, particularly when employees can see clear productivity benefits. Uncontrolled adoption, however, creates risks involving confidentiality, intellectual property, factual accuracy and undocumented decision-making.

A better approach is to define acceptable uses, approved tools, information-handling rules and review expectations. AI literacy should become part of professional competence rather than an isolated technology initiative.

Keeping our eyes open

AI is neither an all-knowing oracle nor merely another software tool. It is a powerful and sometimes unpredictable amplifier.

It can amplify expertise, but it can also amplify misunderstanding. It can improve communication, but it can make weak reasoning sound authoritative. It can free us from routine work, but it can also tempt us to stop examining that work carefully.

The question is therefore not simply whether we should say yes or no to AI. The more useful questions are where it adds value, what could go wrong and which decisions must remain visibly and demonstrably human.

AI! AI! Oh? Is it friend, is it foe?The truth is determined by what we bestow.For wisdom is human, though algorithms grow—So, ‘Aye’ to AI—but with eyes open. Oh!