A team of doctors and researchers wants to use a technique often linked to AI misuse to make AI tools more accurate. Their idea is to place hidden instructions inside medical studies so AI systems read the findings correctly.
The proposal appeared in a commentary published August 24 in npj Digital Medicine. It was led by Nicholas Peoples, a resident physician in emergency medicine at Massachusetts General Hospital and a Duke Kunshan University alumnus.
“AI is unusually prone to providing wrong answers that sound right and are delivered with enormous confidence,” Peoples said. He warns this could affect patient care as more people use AI for medical information.
His co-authors are Bing Luo and Lijing Yan of DKU, along with Ken Milne. Milne is an emergency medicine physician affiliated with the University of Oxford and Western University in Canada.
Turning Prompt Injection Into a Safeguard
The idea came from Peoples’ interest in “prompt injection.” This technique hides instructions in documents to change how an AI system responds, often to mislead it.
Peoples asked whether researchers could use the same method in reverse. Instead of tricking AI, hidden text could guide it on how to read a study.
The team calls this a “SIMcard,” short for Structured Interpretive Metadata. It would give AI systems standard, machine-readable details on a study’s design, main findings, limits and conclusions the evidence cannot support.
The information would sit in a study’s metadata, which today mostly helps computers find papers. Researchers would prepare it, and journal reviewers would check it along with the manuscript.
Luo gave an example. An observational study might link diet soda to stroke, but an AI tool could wrongly say diet soda causes stroke. A SIMcard would help AI separate association from cause.
“It should be understood as a guardrail, not a guarantee,” Luo said. The team says the goal is not to make AI perfect.
The second idea, called FACTS, is for human readers such as journalists. Short for “Framing Appropriate Claims & Their Scope,” it would explain a study’s main claim, context, key limits and readings to avoid.
How One Letter Was Misread for Decades
The authors say the problem is older than AI. In a 2023 article in The BMJ, Peoples, Yan and Duke University professor Truls Østbye described how bad citations spread false claims.
One case involved a 1980 letter in the New England Journal of Medicine. It said addiction was uncommon among hospital patients given opioids for short periods. It was later cited as proof that addiction was rare with long-term use.
A 2017 analysis found 608 citations of the letter, and many left out the hospital setting. Those researchers suggested the misleading citations may have helped fuel the North American opioid crisis.
The authors say AI could speed up this kind of error as people rely on automated summaries. They also point out that most biomedical research comes from high-income countries and is published in English.
“AI has the potential to make medical research more accessible to people around the world. But access alone is not enough,” Yan said. The paper notes that a study done in one population may not apply to patients elsewhere.
Both frameworks are still proposals and have not been tested. The authors say more research is needed across different studies, AI models and prompts. Wide use would also require coordination among researchers, journals, indexing services and AI developers.
