A Pilot Project Leveraging Large Language Models for Automated Screening and Variable Extraction in Observational Studies
Abstract
BACKGROUND: Systematic reviews of observational studies are central to causal inference in chronic disease epidemiology but are increasingly limited by the scale of the literature and heterogeneity in confounder control. There is a need for transparent, open methods that reduce screening burden and make reported exposures, outcomes, and covariates comparable across studies.
OBJECTIVE: To develop and evaluate modular LLM-based pipelines, LitScreen and VarEx, that automate study screening and variable extraction for observational systematic reviews across multiple use cases, including hypertension as a primary exposure with Alzheimer's disease and related dementias (ADRD) as outcomes, and posttraumatic stress disorder (PTSD) as the exposure with self-harm, self-injury, and suicidality as outcomes.
METHODS AND MATERIALS: We built an end-to-end workflow in which reproducible MEDLINE via Ovid queries yield RIS corpora that are processed by LitScreen, a three-phase screening pipeline combining abstract-level evidence extraction, criterion-wise inclusion adjudication with high-recall gates, and full-text retrieval-augmented verification. Screened-in articles enter VarEx, a retrieval-augmented extraction pipeline that identifies role-specific passages and performs evidence-grounded extraction and semantic classification of exposures, outcomes, and covariates into predefined categories aligned with Metaconfoundr. Performance was evaluated on six labeled SYNERGY datasets and…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.