Performance of ChatGPT in Ophthalmic Registration and Clinical Diagnosis: Cross-Sectional Study
Abstract
Background Artificial intelligence (AI) chatbots such as ChatGPT are expected to impact vision health care significantly. Their potential to optimize the consultation process and diagnostic capabilities across range of ophthalmic subspecialties have yet to be fully explored. Objective This study aims to investigate the performance of AI chatbots in recommending ophthalmic outpatient registration and diagnosing eye diseases within clinical case profiles. Methods This cross-sectional study used clinical cases from Chinese Standardized Resident Training–Ophthalmology (2nd Edition). For each case, 2 profiles were created: patient with history (Hx) and patient with history and examination (Hx+Ex). These profiles served as independent queries for GPT-3.5 and GPT-4.0 (accessed from March 5 to 18, 2024). Similarly, 3 ophthalmic residents were posed the same profiles in a questionnaire format. The accuracy of recommending ophthalmic subspecialty registration was primarily evaluated using Hx profiles. The accuracy of the top-ranked diagnosis and the accuracy of the diagnosis within the top 3 suggestions (do-not-miss diagnosis) were assessed using Hx+Ex profiles. The gold standard for judgment was the published, official diagnosis. Characteristics of incorrect diagnoses by ChatGPT were also analyzed. Results A total of 208 clinical profiles from 12 ophthalmic subspecialties were analyzed (104 Hx and 104 Hx+Ex profiles). For Hx profiles, GPT-3.5, GPT-4.0, and residents showed comparable…
How strong is the evidence?
Evidence Hub reads this as a cross-sectional, done in people. Its evidence level is 4 of 5 (Low · observational). Observational designs can show an association but not prove cause.
Read the paper
Related studies
- Sarcopenia, sarcopenic overweight/obesity and risk of cardiovascular disease and cardiac arrhythmia: A cross-sectional study
- Workplace Violence Is Associated With Impaired Work Functioning in Nurses: An Italian Cross-Sectional Study
- Vagus Nerve Dysfunction in the Post-COVID-19 Condition: a pilot cross sectional study
- Association Between Metabolic Associated Fatty Liver Disease and Chronic Kidney Disease: A Cross-Sectional Study from NHANES 2017–2018
- Efficacy and safety of solifenacin succinate 10 mg once Daily: A multicenter, phase III, randomized, double-blind, placebo-controlled, parallel-group trial in patients with overactive bladder
- Association between triglyceride glucose‐body mass index and hypertension in Chinese adults: A cross‐sectional study
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.