Microsoft's AI Diagnostic Tool Outperforms Doctors in Accuracy Tests

Image: Illustrative image · Microsoft Corporation · Public domain · Source

Microsoft has developed an AI diagnostic system that reportedly diagnoses conditions up to four times more accurately than human physicians in tests, signaling a major advance toward AI-driven healthcare.

Microsoft has announced a significant advance in healthcare AI with its Diagnostic Orchestrator tool, which the company claims can diagnose medical conditions substantially more accurately than human doctors. According to research published in the journal Science and reported by The Verge, this AI system achieved diagnostic accuracy around four times higher than physicians in controlled testing scenarios.

The Diagnostic Orchestrator operates by simulating a panel of medical experts, engaging multiple AI agents that collaboratively analyze symptoms and patient histories, determine necessary tests, and reach diagnoses. This procedure mirrors the investigative approach doctors take when diagnosing patients.

In evaluations involving cases sourced from the New England Journal of Medicine, the AI's accuracy was approximately 83%, compared to about 36% for generalist physicians consulting reference materials and 22% for those who did not. Additionally, in emergency department cases from the Beth Israel Deaconess Medical Center's patient database, the AI scored 80% accuracy while human physicians averaged 47%. The AI also tended to provide diagnoses at lower estimated costs.

Powered by OpenAI's GPT-5, the Diagnostic Orchestrator outperformed other leading AI models tested in the study, though newer models such as GPT-6 Astra and others were not yet included. The system was initially developed when GPT-4.1 was state-of-the-art, and subsequent testing suggests that combining multiple AI agents remains beneficial despite improvements in individual model capabilities.

Microsoft acknowledges that real-world clinical validation is still needed, as the tool has so far only been tested retrospectively on past medical cases. The company plans to begin hospital-based trials soon, working with Beth Israel Deaconess Medical Center to assess performance in everyday healthcare settings.

This development forms part of Microsoft's broader ambition for "medical superintelligence," aiming to create AI systems that outperform the best human healthcare providers. Besides diagnosis, Microsoft identifies other crucial healthcare areas for AI, including care coordination and treatment decisions.

While commercialization details and precise timelines remain unclear, Microsoft stresses the importance of a responsible rollout due to strict medical regulatory requirements. The company envisions AI augmenting doctors by reducing routine tasks and increasing time spent with patients.

Microsoft's AI health lead, Dr. Dominic King, and AI CEO Mustafa Suleyman emphasize a human-centered vision where clinicians maintain a central role, even as AI systems reach performance levels comparable to or exceeding human experts. Suleyman projects that intelligent systems like this will soon transform healthcare delivery significantly.