But AI told me so
Medical knowledge advances at a pace difficult for humans to assimilate, and the near-impossibility of staying current is one of medicine's lifelong pressures.
The promise of these tools is perhaps greatest for generalists, junior doctors and those in under-resourced settings, where AI-assisted guidance could meaningfully offset the burden of limited specialist access.
This is almost certainly the first of many waves of AI that will redefine medical practice. The potential is genuine: greater healthcare equity, more timely evidence-based care, and a shift away from static knowledge resources toward advice that can be tailored to the patient at the bedside.
But what happens if AI gets it wrong?
The natural corollary of putting clinically relevant information at a doctor's fingertips is that they will act on it. That is, after all, the point. By design, these platforms push clinicians to the edges of their knowledge comfort zone and convert that extended reach into real-time clinical decisions.
The stated safety backstop is referencing, and with it the promise that the chain of epistemic accountability remains intact. Every AI recommendation is linked to a citable source that has passed through the traditional governance mechanisms of human peer review. Except it requires the clinician to audit it. The same clinician whose efficiency the tool is maximising and whose time is being saved from delving into primary sources. Routine verification is unlikely in the flow of busy point-of-care medicine, and the design of some tools does not necessarily invite it.
A doctor using an AI evidence tool is simultaneously being handed an answer and implicitly asked to verify it. That tension sits at the heart of the problem.
The gap between delivery and verification of the answer is where the risk exists. And it appears to belong to the clinician.
So who wears the clinical negligence liability when the tool gets it wrong?
The highest risk is advice that sits just beyond the clinician's knowledge base, couched among otherwise sensible recommendations, confident in tone, and wrong in a way that isn't ludicrous enough to trigger clinician suspicion. The kind of wrong that gets acted on.
Consider the junior ward doctor, managing an incidental mild hypokalaemia at 11pm, who asks an AI tool for guidance on intravenous potassium replacement. The answer is detailed, referenced and confident. But the dose suggested is appropriate for critical replacement via a central line in intensive care, not an unmonitored ward patient. The doctor, at the edge of their knowledge and reassured by an answer that sounds authoritative, acts on it.
When patient harm results, the tool's defence is already written into its terms of service: the clinician could have, should have, followed the references back to the primary source and cross-referenced against others, identified the hallucination or the contextual inaccuracy, and disregarded it. But the trap is structural. The tool's entire value proposition is that you don't need to interrogate primary sources. Its liability disclaimer depends on the assumption that you will.
It is arguable that these failures will be rare with minor resulting harm, and that the net benefit to patient care (and clinician workload) is significantly positive. This is exciting technology and the clinicians using it are doing so in good faith, motivated by a desire to deliver better care to their patients.
But I think a medico-legal risk does exist for clinicians using these tools. A review of the publicly available terms of service across AI clinical evidence tools currently in widespread use reveals a common fundamental: absolute disclaimers of clinical liability. It is a sector-wide liability architecture, in which the tool is marketed as a trusted clinical co-pilot, and the risk is allocated to the clinician who uses it.
Many doctors will accept these terms at onboarding without reading them. As a profession we should be very mindful about this: embracing a new era of AI-enabled clinical care, while remaining alert to the ways these tools may quietly expose clinicians to unexamined risk, and recognise that the confidence these tools project is not matched by the liability that they accept.
Comment from Avant |
Many AI tools, particularly those falling outside the Therapeutic Goods Administration's framework for software as a medical device, are unregulated so you will need to do your own due diligence to ensure the tool is fit-for-purpose. When you use AI in a clinical context, once you accept AI's recommendations, the advice is yours. It will not be a defence to say, “AI told me so”. Issues around indemnity clauses and disclaimers are complex and they can have implications for insurance cover. If the AI provider's terms and conditions include an indemnity clause or disclaimer, consider seeking advice before agreeing to it. |
More information
For medico-legal advice, please call 1800 128 268, 24/7 in emergencies.
This article was originally published in Connect magazine issue 26 in May 2026.
The information in this publication does not constitute legal, financial, medical or other professional advice and should not be relied upon as such. It is intended only to provide a summary and general overview on matters of interest and it is not intended to be comprehensive. Persons implementing any recommendations contained in this publication must exercise their own independent skill or judgement and seek appropriate professional advice relevant to their own particular circumstances. Compliance with any recommendations will not in any way guarantee discharge of the duty of care owed to patients and others coming into contact with the health professional or practice. Avant and its related entities are not responsible to any person for any loss suffered in connection with the use of this information. Information is only current at the date initially published.