Microsoft AI announced that its Diagnostic Orchestrator system outperformed generalist doctors in diagnosing complex medical cases, achieving 83% accuracy compared to physicians' 36% in a study published on October 22, 2026 in Science. The multi-agent system models a panel of experts to order tests and reason sequentially. Microsoft plans to test the technology in real-world clinical evaluations at Beth Israel Deaconess Medical Center and is training a foundation model with Mayo Clinic.
Diagnostic performance on benchmark cases
- ▪In a study published on October 22, 2026 in Science, Microsoft's Diagnostic Orchestrator system correctly diagnosed complex New England Journal of Medicine cases 83% of the time, compared to 36% for generalist physicians with reference materials and 22% without
- ▪Microsoft's Diagnostic Orchestrator achieved an 80% accuracy score on emergency department patient cases from Beth Israel Deaconess Medical Center, compared to an average diagnostic accuracy of 47% for human physicians
- ▪Microsoft evaluated Diagnostic Orchestrator using more than 300 unusual medical case dilemmas, compiling these cases into a benchmark named SDBench that the company plans to open-source
Evaluating diagnostic work-up efficiency
- ▪Microsoft's study published on October 22, 2026 found that diagnostic work-ups organized by its Diagnostic Orchestrator system reached diagnoses at lower estimated costs than work-ups conducted by human doctors
- ▪Harsha Nori, head of AI at Microsoft AI Health, stated that Microsoft evaluated how judiciously Diagnostic Orchestrator conducts diagnostic work-ups alongside measuring diagnostic accuracy
Diagnostic Orchestrator multi-agent architecture
- ▪Unlike traditional medical AI models that analyze all patient symptoms and lab results simultaneously, Microsoft's Diagnostic Orchestrator updates working hypotheses step-by-step based on initial physical exam and patient history data
- ▪Microsoft's Diagnostic Orchestrator uses multiple AI agents acting like a panel of medical experts to question patients, order tests, and debate diagnoses sequentially
Enhancements to underlying AI models
- ▪On complex medical cases, Microsoft's Diagnostic Orchestrator increased OpenAI's GPT-5.5 accuracy from 72% to nearly 87%, though it yielded no diagnostic accuracy gains on emergency department cases
- ▪Microsoft's Diagnostic Orchestrator increased the diagnostic accuracy of Google's Gemini 2.5 Flash model from 46.4% on its own to nearly 69% when using the multi-agent system
- ▪Microsoft tested Diagnostic Orchestrator using underlying AI models from OpenAI, Google, Anthropic, Meta, and DeepSeek, achieving its highest diagnostic accuracy with OpenAI's GPT-5 and GPT-5.5 models
- ▪On complex medical cases, OpenAI's GPT-5 scored 78.9% accuracy at an estimated cost of $8,400 per case, while Microsoft's Diagnostic Orchestrator reduced costs by about $3,000 per case while raising accuracy by four percentage points
Clinical evaluation at Beth Israel Deaconess
- ▪Google and Beth Israel Deaconess Medical Center published a clinical study in The Lancet evaluating Google's AMIE conversational diagnostic AI during real-world urgent care visits
- ▪Microsoft is partnering with Beth Israel Deaconess Medical Center in Boston to test its Diagnostic Orchestrator's clinical reasoning and triage capabilities in real-world emergency department settings
- ▪Under Microsoft's evaluation trial with Beth Israel Deaconess Medical Center, consenting patients will receive standard care from human clinicians while Microsoft's Diagnostic Orchestrator generates parallel medical recommendations for comparative analysis
Vision for medical superintelligence
- ▪Microsoft AI CEO Mustafa Suleyman defined medical superintelligence as the capacity to surpass the strongest contemporary human-led health systems in medical reasoning and coordination across the full arc of care
- ▪Microsoft AI CEO Mustafa Suleyman predicted that AI performance indistinguishable from human capability on medical superintelligence will arrive within a couple of years
Microsoft's healthcare product portfolio
- ▪Microsoft is collaborating with the Mayo Clinic to train a healthcare foundation model with a planned release in 2027 to enable predictive streaming of patient medical records
- ▪Microsoft currently sells dictation and radiology tools to clinicians and views its Diagnostic Orchestrator as an expansion of its healthcare product portfolio
Story comments
Loading comments…