Microsoft’s diagnostic AI beat doctors in a benchmark. Can that translate to patient care?
A Science study reports higher accuracy on simulated cases. The results do not establish safe, autonomous diagnosis in a hospital.
A Microsoft-led study has reported strong diagnostic results for an AI system that asks questions and selects tests step by step. The result is significant because diagnosis is a sequence of decisions—not simply a quiz—but its setting remains an evaluation based on case records, not a demonstration of autonomous care for live patients.
How the test worked
The paper in Science describes the Sequential Diagnosis Benchmark and the MAI Diagnostic Orchestrator, or MAI-DxO. The system simulates a panel of physicians, revises possible diagnoses and requests further information before choosing an answer.
The benchmark transformed 304 New England Journal of Medicine clinicopathological cases and 325 emergency-department-to-admission encounters from MIMIC-IV-ED into interactive, stepwise encounters. This design tests how a system chooses evidence, as well as whether its final diagnosis is correct.
What the accuracy figures actually compare
The study reported 83.2 percent accuracy on the journal’s complex cases, compared with 36.1 percent for participating generalist physicians using reference material. On emergency-department cases, the corresponding reported results were 80.3 percent for the unconstrained system and 46.8 percent for generalist physicians.
Those figures describe specific evaluation conditions, not the performance of every doctor or every AI model. The paper also reported cost trade-offs: the unconstrained high-accuracy configuration was not the cheapest option. Lower-cost configurations achieved different accuracy levels. A claim that the most accurate system was always cheaper would leave out that distinction.
The distance between a benchmark and a bedside
A retrospective case-based benchmark cannot settle every question about real clinical use. Patient communication, incomplete records, physical examination, changing symptoms and the consequences of a wrong recommendation all matter beyond the final answer score.
The work offers a reason to investigate clinical support tools, not an instruction for readers to replace medical care with a chatbot. Earlier versions of this research circulated before the October 22 journal publication; this report uses the peer-reviewed paper’s figures rather than mixing them with numbers from an older preprint.
