The Uncomfortable Truths About AI in Clinical Practice.
- May 8
- 8 min read
Health, AI & Humanity
HEMI Health: Your AI Companion in Complex Clinical Decisions
By Dr Ezam Mat Ali, FRCPCH BMBS MA (Technology in Clinical Practice)
CEO of MedPlanner
Key Highlights
WhatsApp is widely used for clinical communication yet it carries significant patient data risks that go largely unscrutinised compared to regulated AI platforms
Junior doctors are already using generic AI the question is not whether to allow it but whether to guide it toward safer, accredited alternatives
The deskilling debate is real though it may be masking something else senior clinicians' reluctance to endorse AI for trainees deserves honest examination
Medical education is already changing universities are now assessing students on how well they use AI, not just whether they used it
Healthcare-grade AI built with medicolegal protection is the rational, responsible choice for clinical settings not a restriction but a safeguard
_________________________________________________________________________
I have been in many meetings about AI in healthcare. Most of them are carefully structured, broadly optimistic, and stay comfortably within the lines.
Recently, I had the privilege of sitting down with a team of senior clinicians and their clinical director at a major public hospital. It was a different kind of conversation entirely. And I mean that as a genuine compliment.
What was scheduled as an hour became two. The room contained a range of views. Some were enthusiastic about AI. Some were cautious. Some were genuinely uncertain. Not everyone agreed. And that, I think, is exactly as it should be. What I found most valuable was not consensus but candour. The willingness to ask hard questions, challenge assumptions, and engage seriously with the complexities of bringing AI into clinical practice.
I want to share some of what emerged from that conversation. Not to single anyone out, but because I believe these questions are universal. They are being asked in hospitals and clinical teams everywhere. And they deserve more public discussion than they currently get.
_________________________________________________________________________
The WhatsApp Problem Nobody Wants to Talk About
One of the first topics raised by the clinical team was data security. Specifically, concerns about AI medical scribing tools and where patient data goes, who owns it, and how it is protected.
These are legitimate, important questions. But here is what struck me.
In the same conversation, it emerged that many of the clinicians in that room and their colleagues across the hospital, routinely use WhatsApp to coordinate patient care. Messages that include enough identifiable information to know exactly which patient is being discussed. Sent on personal devices. Stored on personal servers. With no formal data governance framework in place whatsoever.
And nobody in the room had flagged that as a problem.
A peer-reviewed scoping review published in the Journal of Medical Internet Research found that of 346 papers examining WhatsApp use in clinical practice, only 16 addressed record keeping or data storage and that most clinicians, while aware of their statutory obligations, showed a general lack of concern about existing privacy and security legislation. WhatsApp does not sign Business Associate Agreements for healthcare use. It does not comply with HIPAA, GDPR, or equivalent data protection legislation for regulated data transmission. Patient information stored on personal devices creates medicolegal exposure that many clinicians do not even realise they carry.
My observation, and I said this openly in the room, is that the scrutiny applied to regulated, healthcare-grade AI platforms is, ironically, far more rigorous than the scrutiny applied to the informal tools that clinicians use every single day without a second thought.
This is not a criticism of the clinicians. It is a systems problem. And it points to something important: the conversation about data security in healthcare needs to start with the tools already in use. Not just the new ones being introduced.
_________________________________________________________________________
Junior Doctors Are Already Using AI. The Question Is Which One.
The second uncomfortable truth that surfaced was this: junior doctors are using ChatGPT, Claude, and other generic AI platforms to support their clinical work. This is not a rumour. It is happening quietly and routinely.
Research supports this entirely. A national cross-sectional study published in PubMed in 2026 found that 88.7% of medical students reported having used ChatGPT, with 21% using it to help write clinical notes and significant numbers using it for differential diagnoses and clinical decision support. A separate study in PLOS Digital Health found that junior medical students were actually more likely to use AI tools than their senior counterparts, precisely because they had not yet fixed their workflows and were more open to new approaches.
This is the reality. Junior doctors are not waiting for permission. They are already there.
And yet, the clinical team I met with expressed significant concern about the prospect of endorsing a regulated, MHRA-certified AI platform as a clinical decision support tool one with medicolegal protection built in and a clinician-in-the-loop principle at its core. The concern about the regulated platform was discussed at length. The concern about junior doctors using generic, unvalidated AI tools on their personal phones was barely raised.
I found this genuinely puzzling. And I said so.
If a junior doctor uses ChatGPT to generate a differential diagnosis, a tool with no clinical governance, no medicolegal framework, no regulatory certification, and no audit trail, and that output influences a clinical decision, what exactly is the medicolegal position of the clinician? Of the institution?
The answer is: deeply unclear. And that ambiguity sits entirely with the doctor.
_________________________________________________________________________
Is the Deskilling Concern Real? Or Is Something Else Going On?
The third and perhaps most interesting moment in that meeting came when senior leaders expressed reluctance to endorse AI use for junior doctors and trainees. The stated reason: that junior doctors would not learn properly if they relied on AI to support their clinical reasoning.
I want to be honest here. This concern has some merit. The research on automation bias, which is the tendency to over-trust AI outputs even when they are wrong, is well documented. A significant randomised clinical trial published in JAMA found no significant improvement in physicians' diagnostic reasoning when given an LLM assistant. A 2025 commentary in Perspectives on Medical Education also noted that AI literacy, not just AI access, is what determines whether AI enhances or undermines clinical development.
But I also want to be equally honest about something else I observed in that room.
Some of the reluctance did not feel like pedagogical concern. It felt more like professional anxiety. A quiet worry, perhaps unconscious, that if AI can produce a well-reasoned differential diagnosis in seconds, the accumulated clinical wisdom of a senior physician becomes less obviously irreplaceable.
The American Medical Association flagged exactly this dynamic in a 2025 policy report, warning that when AI tools are not transparent or explainable, a clinician's training and expertise can be "removed from decision-making." But the solution is not to ban juniors from using AI. It is to ensure they use it in a way that keeps their clinical judgment central and not replaced by it.
A consultant's value is not their ability to generate a list of diagnoses. It is their ability to synthesise a patient's full story, weigh competing risks, communicate uncertainty, and make a call under pressure. No AI replaces that. But if we are honest, the concern that it might is real and it deserves a far more open conversation than it usually gets.
_________________________________________________________________________
The Way Learning Is Changing, Whether We Endorse It or Not
The final observation I want to share is perhaps the most forward-looking.
Universities are already adapting to this reality. In 2025, 88% of UK university students reported using generative AI tools to complete their assessments up from 53% the previous year. Institutions from Oxford to ETH Zurich to institutions across Southeast Asia are no longer asking whether students use AI. They have already begun redesigning their assessments to evaluate how well students use it.
At an increasing number of universities, students are now assessed on their prompt engineering skills their ability to ask AI the right questions, to evaluate the outputs critically, and to apply the results with judgment and integrity. Durham University has embedded prompt engineering into its curriculum as a core competency. The logic is straightforward. The skill of the future is not knowing things. It is knowing how to interrogate the tools that know things and, just as importantly, knowing when not to trust them.
This shift is inevitable in medicine too. The question is whether healthcare institutions get ahead of it or scramble to catch up after it has already happened on the wards.
_________________________________________________________________________
What I Think We Should Actually Do
I am not writing this to criticise the clinicians I met. They were thoughtful, experienced, and genuinely grappling with real questions. The conversation was one of the most honest I have had about AI in clinical settings.
But I am writing it because I think the current approach, where informal tools are tolerated and regulated tools are scrutinised, is exactly backwards. And because I think there is a better way.
On data security: Clinicians and health leaders need to apply the same rigour to informal communication tools that they apply to regulated clinical AI. If data security is the concern, WhatsApp should be the first conversation. Not the last.
On junior doctors and AI: The rational, responsible response is not to prohibit AI use by trainees. It is to channel that use toward healthcare-grade, accredited platforms tools built with medicolegal protection at their core, where every output is reviewed and owned by the clinician, and where a proper audit trail exists if things go wrong.
At HEMI Health, medicolegal protection is not a feature. It is the founding principle. Every note generated, every decision supported, every output produced is reviewed, approved, and owned by the clinician. That is exactly the kind of AI framework that should be endorsed for trainees. Not because it removes clinical judgment. Because it protects it.
On the deskilling debate: The answer is AI literacy, not AI prohibition. Junior doctors who learn to use AI well who understand its limitations, who interrogate its outputs, who keep their own reasoning primary will be better clinicians for it. Those who use it uncritically, on unregulated platforms, with no governance framework, will not.
On medical education: The shift is already happening. Clinical training programmes that embed AI literacy including how to use accredited clinical AI tools safely and effectively will produce graduates who are ready for the healthcare environment they are actually entering. Those that do not will produce graduates who improvise, unsupported, from day one.
_________________________________________________________________________
A Final Thought
The conversation I had in that hospital meeting room, which ran two hours longer than anyone planned, was the kind of conversation that needs to happen in every hospital, every medical school, and every clinical team across the world.
The tools are here. The junior doctors are already using them. The question is no longer whether AI enters clinical practice. It is whether we shape that entry thoughtfully, with the right governance, the right protection, and the clinician firmly at the centre, or whether we let it happen by default on personal devices through platforms that were never built for medicine.
I know which one I would choose.
_________________________________________________________________________
Dr Ezam Mat Ali is the Founder and CEO of MedPlanner and the creator of HEMI Health, a clinical AI platform built with medicolegal protection as its founding principle. He writes about AI in healthcare, clinical governance, and the future of digital medicine.
If this newsletter resonated with you, please share it with a colleague. And if you would like to explore how HEMI Health supports clinicians safely and responsibly, visit hemihealth.ai.
_________________________________________________________________________
References
Ponathil A et al. WhatsApp in Clinical Practice: The Challenges of Record Keeping and Storage. JMIR mHealth and uHealth. 2021. PMC8708459.
Tangadulrat P et al. Using ChatGPT for Clinical Practice and Medical Education: Cross-Sectional Survey of Medical Students' and Physicians' Perceptions. JMIR Medical Education. 2023. doi:10.2196/50658
Duszak A et al. Medical Student Experiences With ChatGPT: National Cross-Sectional Study. PubMed. 2026. PMID: 41802232.
Zheng L et al. ChatGPT and Large Language Models (LLMs) Awareness and Use: A Prospective Cross-Sectional Survey of U.S. Medical Students. PLOS Digital Health. 2024. doi:10.1371/journal.pdig.0000596
Mahajan A, Bates DW. Clinical Artificial Intelligence: The Case for a New Physician Role. The Lancet Regional Health Americas. 2025. doi:10.1016/j.lana.2025.101280
Turner M et al. Not Replaced but Reinvented: AI Education Pathways to Prepare Future Physicians to Lead Healthcare Transformation. Perspectives on Medical Education. 2025. PMC12636280.
iatroX Clinical AI Insights. The Deskilling Dilemma: Will Clinical AI Erode or Enhance Medical Expertise? 2025.
Programs.com. The Latest AI in Education Statistics. 2026.
Thesify. Generative AI Policies at the World's Top Universities: October 2025 Update. 2025.
American Medical Association. Policy Report on AI Transparency in Clinical Decision-Making. 2025.
Comments