Accuracy of large language models in answering urological questions on lower urinary tract symptoms: comparison with the EAU 2025 guidelines
Abstract
Introduction: The aim of the study was to evaluate the accuracy of responses from three common artificial intelligence (AI) tools - ChatGPT, Claude, and DeepSeek - to patient enquiries regarding lower urinary tract symptoms (LUTS), comparing these responses to European Association of Urology (EAU) 2025 guidelines. As patients increasingly turn to large language models (LLMs), it is crucial to assess their reliability.
Material and methods: Prospective face-to-face and telephone general urology clinics were conducted between April and May 2025, during which consented patients were asked to state the questions they would submit to an AI tool if they were to enquire about their LUTS. These questions were submitted to ChatGPT (GPT-4), Claude (Sonnet 4.0), and DeepSeek (V3), and the responses were summarised and compared against the EAU guidelines. Each response was categorised as correct, missing key elements from the guidelines, or incorrect/misleading.
Results: Sixteen patients participated in the study, and following removal of duplicate questions, a total of 13 were included for analysis. These questions covered symptom causation, diagnostic workup and management of LUTS. All models provided correct information in 92.3% of their responses when compared to the EAU guidelines. However, 92.3-100% of answers omitted key elements from the guidelines, and 30.8-92.3% contained incorrect or misleading content.
Conclusions: Current LLMs provide readily accessible guidance on LUTS;…
Read the paper
Related studies
- Erectile dysfunction
- Testosterone protects from metabolic syndrome-associated prostate inflammation: an experimental study in rabbit
- Alpha1-adrenoceptor subtypes and lower urinary tract symptoms
- Cyclic nucleotide phosphodiesterases: important signaling modulators and therapeutic targets
- Chronic bladder ischemia and oxidative stress: New pharmacotherapeutic targets for lower urinary tract symptoms
- The role of prostate inflammation and fibrosis in lower urinary tract symptoms
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.