Raging against the Machine: The Human Side of the Story
OVERVIEWA real-time diagnostic comparison at Harvard Medical Grand Rounds in September 2023 evaluated GPT-4 and an experienced internist simultaneously assessing a diagnostic unknown before a live audience, providing insights into AI performance in authentic clinical reasoning scenarios. Real-Time Performance: GPT-4 demonstrated notable diagnostic accuracy and often paralleled the internist's clinical reasoning in real time. The comparison revealed that general-purpose language models can engage in sophisticated diagnostic hypothesizing and differential reasoning that approximates experienced physician performance. Clinical Context Matters: The study highlights that GPT-4's performance was strongest when diagnostic reasoning could be grounded in clear clinical presentations and medical literature. Performance limitations emerged in areas requiring nuanced clinical judgment, integration of subtle clinical findings, or navigation of ambiguous presentations where physician experience provides decisive advantage. Implications for AI-Human Collaboration: The findings support models of AI-human collaboration rather than AI replacement. GPT-4 performs well as a diagnostic reasoning partner, particularly in generating differential diagnoses and retrieving relevant medical knowledge, while physicians provide essential functions in clinical judgment, patient communication, and contextual decision-making. ---
Subscribe to read full wiki analysis
Full wiki content — including source provenance, contradiction tracking, related pages, and analyst notes — is available to Subscriber members.
View Plans & SubscribeNew users get a 7-day trial — no credit card required.