The Use of Generative AI for Scientific Literature Searches for Systematic Reviews: ChatGPT and Microsoft Bing AI Performance Evaluation
Abstract
Background: A large language model is a type of artificial intelligence (AI) model that opens up great possibilities for health care practice, research, and education, although scholars have emphasized the need to proactively address the issue of unvalidated and inaccurate information regarding its use. One of the best-known large language models is ChatGPT (OpenAI). It is believed to be of great help to medical research, as it facilitates more efficient data set analysis, code generation, and literature review, allowing researchers to focus on experimental design as well as drug discovery and development.
Objective: This study aims to explore the potential of ChatGPT as a real-time literature search tool for systematic reviews and clinical decision support systems, to enhance their efficiency and accuracy in health care settings.
Methods: The search results of a published systematic review by human experts on the treatment of Peyronie disease were selected as a benchmark, and the literature search formula of the study was applied to ChatGPT and Microsoft Bing AI as a comparison to human researchers. Peyronie disease typically presents with discomfort, curvature, or deformity of the penis in association with palpable plaques and erectile dysfunction. To evaluate the quality of individual studies derived from AI answers, we created a structured rating system based on bibliographic information related to the publications. We classified its answers into 4 grades if the title…
How strong is the evidence?
Evidence Hub reads this as a systematic review. Its evidence level is 1 of 5 (Highest · pooled evidence). Pooled evidence from several studies sits at the top of the evidence pyramid.
Read the paper
Related studies
- Protective effects of tadalafil and N-acetyl cysteine therapy on cisplatin-induced testicular toxicity
- Sildenafil as a Candidate Drug for Alzheimer’s Disease: Real-World Patient Data Observation and Mechanistic Observations from Patient-Induced Pluripotent Stem Cell-Derived Neurons
- Ozone-modified properties of pumpkin seed oil as anti-H. pylori, anticancer, anti-diabetic and anti-obesity agent
- Effectiveness of Low-Intensity Extracorporeal Shock Wave Therapy in Erectile Dysfunction: An Analysis of Sexual Function and Penile Hardness at Erection: An Umbrella Review
- Molecular Docking of Silymarin as a Ligand for Aromatase Enzyme and HIV-1 Reverse Transcriptase Using Molegro Virtual Docker
- Platelet Membrane-Encapsulated Poly(lactic-co-glycolic acid) Nanoparticles Loaded with Sildenafil for Targeted Therapy of Vein Graft Intimal Hyperplasia
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.