GPT-4o and DeepSeek-V3 responses to common migraine questions: a cross-sectional comparative study of accuracy, completeness, empathy, readability and safety
Abstract
Abstract Background People increasingly use large language models to obtain health information. Migraine is common, but some symptoms may overlap with stroke and other secondary headache disorders. We compared the answers given by GPT-4o and DeepSeek-V3 to common migraine questions, with attention to accuracy, completeness, empathy, readability and clinical safety. Methods In this cross-sectional comparative study, 100 patient-facing questions were compiled from Google Trends, the public r/migraine community on Reddit, and patient information resources. Each question was submitted unchanged to GPT-4o and DeepSeek-V3 in separate sessions. Two neurologists, blinded to model identity, independently rated accuracy, completeness and empathy on 1–5 scales and safety on a 0–5 structured rubric. The Ateşman readability score was calculated once from each response. Exact agreement and quadratic-weighted kappa were used for inter-rater agreement. Question-level mean ratings were compared with paired Wilcoxon signed-rank tests, with Holm correction. Results Exact agreement ranged from 88% to 96%, and quadratic-weighted kappa values ranged from 0.810 to 0.920. DeepSeek-V3 had higher accuracy scores than GPT-4o (4.54 ± 0.49 vs 4.37 ± 0.50; Holm-adjusted p = 0.026), higher completeness scores (3.98 ± 0.44 vs 3.46 ± 0.61; adjusted p
How strong is the evidence?
Evidence Hub reads this as a cross-sectional, done in people. Its evidence level is 4 of 5 (Low · observational). Observational designs can show an association but not prove cause.
Read the paper
Related studies
- Calcium Ions in the Physiology and Pathology of the Central Nervous System
- AI-powered in silico twins: redefining precision medicine through simulation, personalization, and predictive healthcare
- In silico functional, structural and pathogenicity analysis of missense single nucleotide polymorphisms in human MCM6 gene
- In Silico drug repurposing pipeline using deep learning and structure based approaches in epilepsy
- Targeting serotonin receptors with phytochemicals – an in-silico study
- In silico exploration of anti-prostate cancer compounds from differential expressed genes
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.