Microsoft AI tool outperforms doctors in complex diagnostic tests, study finds

Microsoft researchers are developing an AI tool designed to replicate doctors’ step-by-step diagnostic process and improve the accuracy and efficiency of medical work-ups. (Image Source: Reuters)

AI Narration

Listen

00:00

Thank you for your feedback.

Your feedback has been reported succesfully.

Microsoft researchers say an AI tool that diagnoses patients step by step, as doctors do, was more than twice as accurate as general physicians at solving complex medical cases, according to a study published on Thursday in Science.

The model identifies and coordinates the questions that should be asked of a patient, as well as the exams and tests that should be ordered, using an “orchestrator” that organises how AI will tackle a problem by employing ⁠a ​simulated panel of clinical experts collaborating to solve diagnostic cases, according to the report.

Existing medical AI models typically take an all-at-once approach that examines all of a patient’s symptoms, lab results and imaging studies to generate a diagnosis.

With the orchestrator coordinating the ​steps ​to reaching a diagnosis, the AI model first ⁠decides what to ask about the patient’s history and what is significant in the physical exam. Based on what it learns, ‌it decides which tests to order, considers the results, and orders further testing as appropriate.

All the while, it is updating its working hypotheses and plans, the research leaders said in an interview.

Tasked with diagnosing more than 300 highly unusual and confusing medical dilemmas, the model employing the diagnostic orchestrator was more than twice as accurate at “working up” the patient and determining the diagnosis ⁠as general physicians using ⁠textbooks and the internet.

The tool also outperformed human physicians when presented with scenarios drawn from a major database of ⁠emergency room admissions.

Story continues below this ad

Assessments comparing ‌these types of older systems to human doctors usually use ​multiple-choice questions, the researchers said.

“AI has become very ‌good at getting a 100% pass rate on those,” said Dr. Dominic King, the corporate vice president of the health unit at Microsoft ‌AI, whose team is ​developing the new ​technology.

“But of ​course, that’s not real-world. No patient ever came to see me with a nice summary.”

The researchers also found that ​diagnostic processes cost less when the orchestrator designs the ⁠work-ups than when human doctors do.

Story continues below this ad

“We’re not just measuring the accuracy of the systems. We’re also measuring how judiciously and carefully it goes through a diagnostic work-up,” said ‌Harsha Nori, ⁠head of AI at Microsoft AI Health.

In a collaboration with Beth Israel Deaconess Medical Center in Boston, the researchers will next ​test the tool’s ability to triage patients in emergency department waiting rooms.