A study presented at the European Respiratory Society Congress in Barcelona found that AI chatbots often give incorrect advice to people with obstructive sleep apnea. In about a third of test cases, the chatbots told patients their symptoms were not serious enough to see a doctor.
Obstructive sleep apnea, known as OSA, is a condition where a person's breathing repeatedly starts and stops during sleep. It often comes with loud snoring and frequent waking. Left untreated, it can raise the risk of high blood pressure, stroke, heart disease and type 2 diabetes.
The research was led by Dr Deeban Ratneswaran, a Research Fellow at Guy's and St Thomas' NHS Foundation Trust in London. He is also a Visiting Academic at King's College London.
Dr Ratneswaran said free AI chatbots now handle hundreds of millions of health-related questions each week. He said most existing research checks whether chatbots answer medical questions correctly, but not how they respond when a patient pushes back.
How the Study Was Designed
The research team built seven realistic OSA patient profiles. Each one met the medical criteria for a referral to a sleep study, which is a test that monitors breathing overnight to diagnose the condition.
Each scenario was run in two versions with the same medical facts. In one version, the patient was open and cooperative. In the other, the patient played down their symptoms and resisted a referral.
The team ran these scenarios through five widely used chatbots: ChatGPT, Google Gemini, Claude, DeepSeek and Grok. In total, they logged 700 conversations.
Chatbot Accuracy Fell Sharply With Resistant Patients
When the patient was cooperative, the chatbots gave correct advice to seek a specialist in all 350 conversations. That is a 100% success rate.
When the same medical facts came from a patient who resisted referral, correct advice held up in only 64% of conversations, or 225 out of 350. Dr Ratneswaran said the advice was abandoned more than a third of the time based purely on how the patient spoke.
The drop was worse in serious cases. In a textbook severe case, correct advice survived only 22% of the time. In a scenario involving a man who had already fallen asleep while driving, correct advice held up in just 32% of conversations, and the driving risk was often left unmentioned.
In roughly a quarter to half of the resistant-patient conversations, depending on the chatbot, the AI offered lifestyle tips instead of recommending a referral. Researchers say this effectively endorsed a delay in treatment.
Dr Ratneswaran advised caution when using chatbots for these symptoms. He said anyone who snores loudly, stops breathing during sleep, or struggles with daytime sleepiness, especially while driving, should see a clinician even if a chatbot suggests waiting.
Dr Io Hui, Chair of the European Respiratory Society's Group on M-health and e-health, was not involved in the research. She said the issue is not what the chatbots know, but how they handle disagreement from a user.
Dr Hui said the tools can be a useful source of information, but they are largely unregulated and could stop people from getting treatment. She said anyone with possible OSA symptoms should speak to a doctor for further advice.
