AI medical models ‘advise treatments based on real-world biases’

Artificial intelligence models may recommend different treatments for the same medical condition based solely on a patient’s socioeconomic and demographic characteristics, researchers warn. The researchers invented nearly three dozen different patients and asked nine healthcare large language AI models how each one should be managed, in a thousand different emergency room situations. Despite identical clinical details, the AI models occasionally altered decisions based on patients’ personal characteristics, affecting priority for care, diagnostic testing, treatment approach, and mental health evaluation, the researchers reported in Nature Medicine .

For example, advanced diagnostic tests such as CT scans or MRI were more often recommended for high-income patients, while low-income patients were more frequently advised to undergo no further testing, somewhat mimicking real-world healthcare inequities. The problems were seen in both proprietary and open-source AI models, the researchers found. “AI has the power to revolutionise healthcare, but only if it’s developed and used responsibly,” study co-leader Dr Girish Nadkarni of the Icahn School of Medicine at Mount Sinai in New York said in a statement.



“By identifying where these models may introduce bias, we can work to refine their design, strengthen oversight, and build systems that ensure patients remain at the heart of safe, effective care,” added co-author Dr Eyal Klang, also of the Icahn School. Meanwhile, researchers are closer to being able to fix the life-altering dryness of the mouth and eyes that afflicts patients with Sjogren’s syndrome, based on success of two approaches tested in mice. The symptoms of the autoimmune disorder can make it hard to speak, eat and sleep.

But exactly how the disease shuts down the body’s production of tears and saliva has been a mystery until now, researchers reported in the International Journal of Oral Science . Their new study found that early in the progression of Sjogren’s syndrome, a protein called tricellulin, which clasps together the cells of the glands that produce tears and saliva, is destroyed. Loss of the tight cellular junctions results in inadequate saliva secretion, the researchers found.

Two possible interventions – an investigational drug (AT1001) and an experimental molecule – each restored saliva secretion in the mice, one by repairing the cell junctions and the other by stopping the breakdown of the junctions before it began. Both restored normal gland function, offering a potential blueprint for human treatment. “This changes how we think about treating Sjogren’s syndrome,” study leader Dr Xin Cong of Peking University said in a statement.

“We’re moving beyond simply calming inflammation. Now we can fix the actual structural damage in the glands,” Dr Xin said. “Both approaches worked.

”.