Assessment of Short-Answer Questions by ChatGPT in a Medical School Course
OVERVIEWA medical school study demonstrates that iterative prompt engineering guided by pedagogical principles and linguistic flexibility yielded moderate concordance between ChatGPT and human graders in assessing short-answer test questions, with implications for AI-driven grading systems in medical education. Grading Automation Potential: AI-driven grading systems have potential to reduce the significant administrative burden of short-answer assessment in medical education. ChatGPT, when properly prompted with pedagogical context and linguistic flexibility, can achieve reasonable concordance with human expert graders. Prompt Engineering as Pedagogy: The study reveals that effective AI grading requires more than technical prompt optimization—it requires grounding prompts in explicit pedagogical principles and allowing for linguistic variability in acceptable answers. This finding highlights the need for educational expertise in AI system design, not just technical expertise. Moderate Concordance Limitations: While the results are encouraging, moderate rather than high concordance suggests that AI grading systems should complement rather than replace human assessment in high-stakes educational contexts. The findings support AI-assisted grading workflows where AI generates initial assessments that educators review and validate. ---
Subscribe to read full wiki analysis
Full wiki content — including source provenance, contradiction tracking, related pages, and analyst notes — is available to Subscriber members.
View Plans & SubscribeNew users get a 7-day trial — no credit card required.