Evaluating ChatGPT Ability to Answer Urinary Tract Infection-Related Questions
Abstract
INTRODUCTION
For the first time, the accuracy and proficiency of ChatGPT answers on urogenital tract infection (UTIs) were evaluated.
METHODS
The study aimed to create two lists of questions: frequently asked questions (FAQs, public-based inquiries) on relevant topics, and questions based on guideline information (guideline-based inquiries). ChatGPT responses to FAQs and scientific questions were scored by two urologists and an infectious disease specialist. Quality and reliability of all ChatGPT answers were checked using the Global Quality Score (GQS). The reproducibility of ChatGPT answers was analyzed by asking each question twice.
RESULTS
All in all, 96.2% of FAQs (75/78 inquiries) related to UTIs were correctly and adequately answered by ChatGPT, and scored GQS 5. None of the ChatGPT answers were classified as GQS 2 and GQS 1. Moreover, FAQs about cystitis, urethritis, and epididymo-orchitis were answered by ChatGPT with 100% accuracy (GQS 5). ChatGPT answers for EAU urological infections guidelines showed that 61 (89.7%), 5 (7.4%), and 2 (2.9%) ChatGPT responses were scored GQS 5, GQS 4, and GQS 3, respectively. None of the ChatGPT responses for EAU urological infections guidelines were categorized as GQS 2 and GQS 1. Comparison of mean GQS values of ChatGPT answers for FAQs and EAU urological guideline questions showed that ChatGPT was similarly able to respond to both question groups (p= 0.168). The ChatGPT response reproducibility rate was highest for the FAQ…
Read the paper
Related studies
- Association of lower urinary tract symptoms and OAB severity with quality of life and mental health in China, Taiwan and South Korea: results from a cross-sectional, population-based study
- Ten-Day Quadruple Therapy Comprising Low-Dose Rabeprazole, Bismuth, Amoxicillin, and Tetracycline Is an Effective and Safe First-Line Treatment for Helicobacter pylori Infection in a Population with High Antibiotic Resistance: a Prospective, Multicenter, Randomized, Parallel-Controlled Clinical Trial in China
- The Role of the NLR and PLR in Urinary Tract Infection
- Threat of Antimicrobial Resistance among Pilgrims with Infectious Diseases during Hajj: Lessons Learnt from COVID-19 Pandemic
- Leclercia adecarboxylata urinary tract infection in a patient with bladder cancer and recurrent hematuria
- A Case Series of Staphylococcus lugdunensis Infection in Cancer Patients at an Academic Cancer Institute in the United States
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.