Microsoft's diagnostic AI scores above doctors on past cases, but has not been tested in clinics
The system has only been run on historical records, and Microsoft has no commercial timeline for it.
Microsoft says its Diagnostic Orchestrator, a multi-agent system built on OpenAI's GPT-5, reached about 83 percent accuracy on complex case reports from the New England Journal of Medicine, against roughly 36 percent for generalist physicians with references. On emergency department cases from Beth Israel Deaconess Medical Center it scored about 80 percent, compared with an average of 47 percent for human doctors. The results were published in Science, and the company says it will open-source its SDBench benchmark. Much of the gain depends on the underlying model: on emergency cases, the system added little over GPT-5 alone, while on complex cases with the newer GPT-5.5 it lifted accuracy from 72 percent to nearly 87 percent. Microsoft plans a real-world evaluation at Beth Israel, but the system has only been tested on past cases and has no commercial timeline.
Why it matters
The results show that a multi-agent system can outscore physicians on written case reports, but that the gain is uneven: on emergency cases it added little beyond the underlying model, while on complex cases it lifted accuracy substantially with a newer model. Patients and clinicians have no evidence yet of how it would perform in live care, so the results mark a promising research step rather than a change in how diagnoses are made. A planned study with Beth Israel Deaconess Medical Center will be the first test against human doctors in a real clinical setting.
Matters or hype?
Does this story matter, or is it hype? Vote before you see what everyone else said.