AI is better at diagnosis than a doctor
The study was devised by Prof. Dan Zeltzer, a digital health expert at TAU’s Berglas School of Economics, and conducted at Cedars-Sinai Connect, a clinic in Los Angeles that provides remote urgent care consultations by video.
The researchers looked at the cases of 461 patients who sought online help with respiratory, urinary, eye, vaginal or dental problems over a month last summer. Patients took part in an AI chat with the clinic’s algorithm before speaking to the physician, who had the final say on their diagnosis and treatment.
The algorithm had access to the patients’ medical records and suggested appropriate treatment, including prescriptions, tests and referrals. A panel of physicians, all with at least 10 years’ clinical experience, then reviewed the recommendations made by both the AI and the MDs, scoring them as optimal, reasonable, inadequate, or potentially harmful.
As well as finding that the AI made more optimal recommendations, the panel also said AI made fewer harmful decisions — 2.8% vs. 4.6% cent of physicians’ decisions. In 68% of the cases, the AI and the physician received the same score; in 21%, the algorithm scored higher than the physician; and in 11% of cases the physician’s decision was considered better.
The researchers concluded that AI scored better because it adheres more strictly to clinical guidelines, identifies relevant information in the medical record more comprehensively, and is more precise at identifying symptoms that could indicate a more serious condition.
Physicians, on the other hand, are more flexible and bring real-world experience to their assessment of a patient. “We found that AI, based on a targeted intake process, can provide diagnostic and treatment recommendations that are, in many cases, more accurate than those made by physicians,” said Zeltzer.
“The relatively common conditions included in our study represent about two-thirds of the clinic’s case volume, and thus the findings can be meaningful for assessing AI’s readiness to serve as a decision-support tool in medical practice.”
Could AI replace doctors altogether? Not exactly.
“We can envision a near future in which algorithms assist in an increasing portion of medical decisions, bringing certain data to the doctor’s attention, and facilitating faster decisions with fewer human errors,” said Zeltzer. “Of course, many questions still remain about the best way to implement AI in the diagnostic and treatment process, as well as the optimal integration between human expertise and artificial intelligence in medicine.”
The study was part-funded by Israeli startup K Health, which jointly operates the Cedars-Sinai Connect clinic. The paper was published in Annals Internal Medicine and was presented at the annual conference of the American College of Physicians (ACP).
John Jeffay