The UK’s National Health Service (NHS) is facing scrutiny as a regulatory body has issued a stark warning about the potential risks posed by AI tools used to transcribe patient-doctor consultations. These artificial intelligence systems, rapidly being integrated into the NHS, have been found to make critical errors, including misidentifying drug and disease names, which could jeopardize patient safety.
Patient Experiences Highlight AI Glitches
One particularly alarming case involved a female NHS healthcare professional who was severely distressed when an AI medical record tool incorrectly summarized her consultation. The AI report wrongly stated she had “demyelination,” a serious neurological condition that can lead to multiple sclerosis. The patient only discovered the error after questioning the AI-generated MRI scan results, which were then corrected to “no demyelination.” “The error was eventually corrected, but it was a very distressing experience for me to be given a wrong diagnosis and then be told it was just a typo,” she recounted anonymously.
Healthwatch England, a patient advocacy group, brought this case and others to light, emphasizing the threat these AI tools pose to patient safety as they become more prevalent within the NHS. Healthwatch warned that if such errors go unnoticed, they could be entered into patient records, leading to incorrect treatment plans.
In another instance, an AI tool confused a prescribed medication with a similarly named one. Again, it was the patient, not the medical professionals, who identified the mistake. A further case involved an AI-generated summary letter omitting crucial advice from a specialist to follow up with a general practitioner for a migraine prescription refill, potentially leaving the patient without necessary medication.
Systemic Issues and Regulatory Concerns
Healthwatch revealed that they have heard from “multiple patients who have identified errors that clinicians have not,” highlighting a critical gap in oversight. “If patients had not identified these errors, this inaccurate information could have remained on their medical records indefinitely,” the organization stated.
Despite these concerns, the UK government’s 10-year plan for the NHS aims to leverage AI to free up clinicians from administrative burdens, allowing them more time for patient care. AI is central to the NHS’s transition from analog to digital services. However, Healthwatch emphasizes the need for clear protocols: “Healthcare will never be error-free. But our findings show there is an urgent need for clarity on how patients should report errors made by AI medical record tools or by clinicians using these tools, and how these errors are rectified.”
Rachel Power, CEO of The Patients Association, commented, “Patient trust and confidence in this technology depend on good communication and genuine patient partnership, both of which are currently lacking.”
A significant point of contention is the UK’s Medicines and Healthcare products Regulatory Agency’s (MHRA) decision not to classify AI medical record tools as medical devices. Healthwatch expressed concern that this means England currently lacks a comprehensive regulatory framework to ensure the safe and effective use of these tools nationwide. Currently, 27 different AI tools are in use by GPs and hospital doctors across England.
Broader AI Accuracy Debates
The accuracy of AI has been a recurring issue. In Rotherham, some patients complained that AI front-desk systems in GP clinics struggled to understand their local Yorkshire accents. Dr. Shier Ziser Dawood, a London GP, previously warned in a medical journal that AI tools could be a “double-edged sword” for GPs. She recalled an instance where an AI system incorrectly stated she had prescribed Prozac to a patient, a drug she neither prescribed nor discussed.
This phenomenon, known as an “AI hallucination,” where the AI fabricates information not present in the consultation, means doctors must meticulously review all AI-generated notes. Dr. Dawood argued that these tools, therefore, do not currently save doctors’ time. She noted that NHS management anticipates AI tools will eliminate the need for doctors to take manual notes, potentially allowing GPs to see two additional patients daily. However, current consultation times for UK GPs are already among the shortest globally.
Dr. Charlotte Blease, an expert in AI applications in healthcare at Uppsala University, Sweden, acknowledged, “AI does make mistakes, and it is making them.” Her research indicates AI errors are more common in consultations with multiple patients or companions, complex medical histories, or when English is not the patient’s first language. Yet, she also stated, “The reality is, doctors make mistakes without AI. And while the error rate might be higher with AI, it’s possible.” In a survey of 1003 UK GPs last year, over half found AI-generated records more accurate than their own manual notes.
