Credit: Unsplash/CC0 Public Domain

Researchers from the University of Oxford have benchmarked artificial intelligence (AI) tools capable of automatically removing personal information from patient electronic health records (EHRs) in a key step toward enabling large-scale, confidential medical research.

As health care becomes digitized, the wealth of information stored in millions of electronic health records (EHRs) is providing a valuable resource. These routinely collected data are driving advances in research, education, and quality improvement. But the increasing interest in using EHRs to train AI models aimed at improving patient outcomes is raising questions over whether current de-identification methods are robust enough to fully protect patient privacy.

"Patient confidentiality is essential to building public trust in health care research," said Dr. Rachel Kuo, NIHR Doctoral Research Fellow at Oxford University. "Manual redaction of personally identifiable information such as patient names or locations is time-consuming and expensive. Automated de-identification could alleviate this burden, but we need to be sure that software can meet an acceptable standard of performance."

The study, published in iScience, was a collaboration among Dr. Rachel Kuo and Professor Dominic Furniss from the Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Dr. Andrew Soltan from the Department of Oncology, and Professor David Eyre from the Big Data Institute. It evaluated the ability of large language models (LLMs) and purpose-built software tools to detect and remove patient names, dates, medical record numbers, and other identifiers from real-world records without altering clinical content.

The first step was to test the ability of a human to anonymize the data. The team manually redacted 3,650 medical records, compared and corrected the data until they had a complete set to use as a benchmark. They then compared two task-specific de-identification software tools (Microsoft Azure and AnonCAT) and five general-purpose LLMs, including GPT-4, GPT-3.5, Llama-3, Phi-3, and Gemma, for redacting identifiable information.

Microsoft’s Azure de-identification service achieved the highest performance overall, closely matching human reviewers. GPT-4 also performed strongly, demonstrating that modern language models can accurately remove identifiers with minimal fine-tuning or task-specific training.

However, the study also revealed risks. Some models produced hallucinations, where text that was not present in the original record was added, occasionally introducing fabricated medical details.

"While some large language models perform impressively, others can generate false or misleading text," explained Dr. Soltan. "This behavior poses a risk in clinical contexts, and careful validation is critical before deployment."

The researchers concluded that automating de-identification could significantly reduce the time and cost required to prepare clinical data for research, while maintaining patient privacy in compliance with data protection regulations.

"This work shows that AI has the potential to be a powerful ally in protecting patient confidentiality," said Professor Eyre. "But human judgment and strong governance must remain at the center of any system that handles patient data."

More information: Rachel Kuo et al, Benchmarking transformer-based models for medical record de-identification in a single center multi-specialty evaluation, iScience (2025). DOI: 10.1016/j.isci.2025.113732

Citation: AI models can rival humans in anonymizing patient information from electronic health records (2025, December 9) retrieved 9 December 2025 from https://medicalxpress.com/news/2025-12-ai-rival-humans-anonymizing-patient.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.