Diagnostic accuracy of large language models for emergency department triage: a systematic review and meta-analysis
Abstract
Objective: Large language models (LLMs) have shown potential in the field of emergency department (ED) triage. However, a comprehensive evaluation of their diagnostic performance and a comparison with triage nurses have not been extensively explored. To systematically evaluate the comparative triage accuracy of LLMs versus triage nurses in ED, assess their performance in identifying patients assigned to the highest-acuity triage category by the reference standard.
Methods: A systematic search was conducted in CNKI, Wanfang, VIP, CBM, PubMed, Embase, Web of Science, and Cochrane Library databases from inception to March 2025. Two researchers independently screened and evaluated the quality of the selected studies and conducted meta-analyses using RevMan 5.4 and Stata 16.0.
Results: A total of 11 studies were included, involving 3088 real or simulated ED cases. The pooled odds ratio (OR) for LLM triage accuracy was 0.65 (95% CI: 0.32-1.34; I² = 97%). For identifying patients assigned to the highest-acuity triage category by the reference standard, LLMs demonstrated a pooled sensitivity of 61% (95% CI: 48%-73%) and specificity of 97% (95% CI: 88%-99%), with statistically significant differences (p < 0.05). The pooled area under the curve (AUC) was 0.82 (95% CI: 0.78-0.85). These estimates reflect concordance with triage category by the reference standard rather than discrimination of outcome-anchored physiological acuity. No significant publication bias was detected by Deeks'…
How strong is the evidence?
Evidence Hub reads this as a meta-analysis, done in people. Its evidence level is 1 of 5 (Highest · pooled evidence). Pooled evidence from several studies sits at the top of the evidence pyramid.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.