Google's AMIE medical AI matches primary care physicians in simulated video consultations

Google Research and Google DeepMind have developed AMIE, a Gemini-powered medical AI system that conducts real-time clinical video consultations. In a randomized study with patient actors, AMIE matched or outperformed 10 primary care physicians, scoring 83% overall compared to the doctors' 68% and achieving 91% diagnostic accuracy. Despite these strong results, researchers emphasize that AMIE remains a research prototype with limitations in detecting subtle physical cues, and requires extensive real-world validation before clinical deployment.

AMIE video consultation system

• ▪The AMIE video consultation system interprets visual and auditory information from patients, guides them through virtual physical examinations, and performs diagnostic reasoning in real time.
• ▪Google Research and Google DeepMind developed AMIE, a medical AI system built on Gemini and Project Astra, to conduct real-time clinical consultations over video.

Multi-agent AI architecture

• ▪AMIE uses an asynchronous multi-agent architecture consisting of a talker agent, a planner agent, and a perception agent to manage dialogue, clinical reasoning, and audio-visual processing.
• ▪The asynchronous multi-agent architecture reduced AMIE's mean turn latency from 21.4 seconds to 2.6 seconds compared with Google's earlier sequential architecture.

OSCE study design

• ▪Google evaluated AMIE in a randomized multi-arm Objective Structured Clinical Examination study comparing AMIE with 10 US board-certified primary care physicians across 100 simulated patient scenarios.
• ▪The OSCE study utilized 15 professional standardized patient actors to portray conditions across five body systems, and a panel of 20 primary care physicians evaluated the consultations.

Clinical performance evaluation results

• ▪AMIE Video achieved a top-ranked diagnostic accuracy of 91% in matching the prespecified reference diagnosis, compared to 77% achieved by the primary care physicians.
• ▪Clinical evaluators gave AMIE Video a higher overall score of 83% compared to 68% for primary care physicians in simulated real-time video consultations.
• ▪Evaluators rated AMIE Video higher on eliciting physical signs and guiding virtual examination maneuvers than both the primary care physician group and text-only AMIE.

Research limitations

• ▪AMIE struggled with subtle perceptual tasks, including detecting nystagmus, tremors, or interpreting patient mood, and occasionally missed visual cues.
• ▪The AMIE study relied on simulated patient actors and prepared scenarios, which cannot fully reproduce the variability, symptoms, and behaviors of real-world clinical encounters.

Future deployment requirements

• ▪Google stated that AMIE remains a research system and requires further studies involving real patients before it can be deployed in real-world clinical settings.
• ▪Google has initiated real-world clinical research using the text-based version of AMIE, including a feasibility study with Beth Israel Deaconess Medical Center and an ongoing study with Included Health.