Are ChatGPT and large language models “the answer” to bringing us closer to systematic review automation?
Abstract
In this commentary, we discuss ChatGPT and our perspectives on its utility to systematic reviews (SRs) through the appropriateness and applicability of its responses to SR related prompts. The advancement of artificial intelligence (AI)-assisted technologies leave many wondering about the current capabilities, limitations, and opportunities for integration AI into scientific endeavors. Large language models (LLM)—such as ChatGPT, designed by OpenAI—have recently gained widespread attention with their ability to respond to various prompts in a natural-sounding way. Systematic reviews (SRs) utilize secondary data and often require many months and substantial financial resources to complete, making them attractive grounds for developing AI-assistive technologies. On February 6, 2023, PICO Portal developers hosted a webinar to explore ChatGPT’s responses to tasks related to SR methodology. Our experience from exploring the responses of ChatGPT suggest that while ChatGPT and LLMs show some promise for aiding in SR-related tasks, the technology is in its infancy and needs much development for such applications. Furthermore, we advise that great caution should be taken by non-content experts in using these tools due to much of the output appearing, at a high level, to be valid, while much is erroneous and in need of active vetting.
How strong is the evidence?
Evidence Hub reads this as a systematic review. Its evidence level is 1 of 5 (Highest · pooled evidence). Pooled evidence from several studies sits at the top of the evidence pyramid.
Read the paper
Related studies
- Challenges and opportunities with drug repurposing: finding strategies to find alternative uses of therapeutics
- Challenges in Chagas Disease Drug Development
- T Cell Specificity: A Great Challenge in Chagas Disease
- Benznidazole treatment leads to DNA damage in Trypanosoma cruzi and the persistence of rare widely dispersed non-replicative amastigotes in mice
- Rapid Molecular Detection of Terbinafine-resistant Dermatophytes
- Identifying novel drugs with new modes of action for neglected tropical fungal skin diseases (fungal skinNTDs) using an Open Source Drug discovery approach
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.