Comparative Performance and Safety of Three Large Language Models in Pediatric Paracetamol Poisoning Management
Abstract
Abstract Background. Paracetamol overdose is a common reason for pediatric poisoning consultations and emergency department visits, and its management is highly time and dose-dependent. Large language models (LLMs) are increasingly being explored for clinical decision support, but evidence on pediatric poisoning is scarce. We compared the performance and safety of three contemporary LLMs for managing pediatric paracetamol poisoning using guideline-based simulated scenarios. Methods. In this cross-sectional, comparative, scenario-based study, 30 hypothetical pediatric paracetamol poisoning scenarios were presented to ChatGPT 5.5, Claude Sonnet 4.6, and Gemini 3 Flash via their public browser-based interfaces, with a standardized single-turn prompt (90 responses). Blinded to model identity, a three-clinician expert panel scored each response by consensus using a study-specific Artificial Intelligence Evaluation Score for Clinical Poisoning Vignettes (AIES-CPV; range 5–25), an eight-item key-action checklist (0–8), the Global Quality Scale (GQS; 1–5), reference-standard concordance, and the presence of critical safety errors, against a predefined reference algorithm derived primarily from the Australia and New Zealand guidelines. Models were compared using the Friedman test (with Wilcoxon signed-rank tests and a Bonferroni correction for pairwise comparisons) and the Cochran Q test for critical error rates. Results. Claude Sonnet 4.6 achieved significantly higher total AIES-CPV…
How strong is the evidence?
Evidence Hub reads this as a cross-sectional. Its evidence level is 4 of 5 (Low · observational). Observational designs can show an association but not prove cause.
Read the paper
Related studies
- Effect of dexamethasone as an analgesic adjuvant to multimodal pain treatment after total knee arthroplasty: randomised clinical trial
- Effect of a Postoperative Multimodal Opioid-Sparing Protocol vs Standard Opioid Prescribing on Postoperative Opioid Consumption After Knee or Shoulder Arthroscopy: A Randomized Clinical Trial
- Comparison of the Post-Caesarean Analgesic Effect of Adding Dexmedetomidine to Paracetamol and Ketorolac: A Randomized Clinical Trial
- A triple-blind randomized clinical trial of different associations between dexamethasone and non-steroids anti-inflammatories for preemptive action in third molar extractions
- An open label randomized clinical trial of Indomethacin for mild and moderate hospitalised Covid-19 patients
- Oral paracetamol and/or ibuprofen for treating pain after soft tissue injuries: Single centre double-blind, randomised controlled clinical trial
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.