Guideline-Integrated Large Language Models Improve Decision Support for Acute Ear, Nose and Throat Emergencies
Abstract
BACKGROUND
Acute otolaryngologic emergencies such as sudden sensorineural hearing loss (SSNHL), epistaxis, and acute facial paralysis frequently present to emergency departments (EDs), where early guideline-concordant decisions are critical to prevent irreversible morbidity. Large language models (LLMs) are increasingly explored for clinical decision support, yet their ability to consistently apply specialty guidelines in time-sensitive settings remains uncertain. Retrieval-augmented generation (RAG), which supplies LLMs with authoritative guideline content during response generation, may improve accuracy and safety.
OBJECTIVES
To determine whether integrating American Academy of Otolaryngology-Head and Neck Surgery (AAOHNS) Clinical Practice Guidelines via RAG improves the accuracy, guideline adherence, and safety of LLM-generated decision support for acute ear, nose and throat (ENT) emergencies in the ED.
METHODS
Twelve standardized clinical vignettes representing ED presentations of SSNHL, epistaxis, and Bell's palsy were developed. Two LLMs (ChatGPT and Gemini) generated responses under baseline conditions and with guideline-integrated RAG. Five blinded otolaryngologists independently evaluated outputs using a nine-domain rubric assessing diagnostic accuracy, diagnostic workup, management quality, referral appropriateness, guideline adherence, and risk of misleading information. Non-parametric tests compared performance, and inter-rater reliability was assessed using…
Read the paper
Related studies
- The Matrisome: In Silico Definition and In Vivo Characterization by Proteomics of Normal and Tumor Extracellular Matrices
- The Escherichia coli MG1655 in silico metabolic genotype: Its definition, characteristics, and capabilities
- In silico pharmacology for drug discovery: methods for virtual ligand screening and profiling
- In silico prediction of splice-altering single nucleotide variants in the human genome
- In silico clinical trials: how computer simulation will transform the biomedical industry
- In silico optimization of a guava antimicrobial peptide enables combinatorial exploration for peptide design
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.