Advancing Evidence-Based Medicine for Population, Intervention, Comparison, and Outcome Element Recognition and Extraction in Medical Literature: Large Language Model Approach
Abstract
Background: The exponential expansion of biomedical literature has created an urgent need for efficient methods to recognize and extract population, intervention, comparison, and outcome (PICO) elements-the foundational elements of evidence-based medicine.
Objective: This study systematically evaluated 2 complementary approaches for automating PICO recognition and extraction in medical literature: prompt engineering optimization and parameter-efficient fine-tuning (PEFT) of large language models (LLMs).
Methods: We developed a dual-phase methodological framework: (1) systematic prompt optimization incorporating in-context learning, chain of thought (COT), and multipath reasoning strategies; and (2) PEFT of the LLM architecture using low-rank adaptation (LoRA), quantized LoRA, and freeze techniques. The PubMed-PICO and NICTA-PIBOSO benchmark datasets were used for recognition tasks, and the EBM-NLP dataset was used for extraction tasks. Performance metrics included precision, recall, and F1-score. F1-score was adopted as the major metric as it balances precision and recall.
Results: For prompt engineering, COT achieved the overall best performance across both recognition and extraction tasks. For example, in the recognition task, COT obtained strong average F1-scores of 77.1% (SD 0.5%) for the population element and 84.5% (SD 0.4%) for the outcome element on PubMed-PICO. In the extraction task, COT achieved the highest average F1-score of 73.9% across 3 PICO elements (the…
Read the paper
Related studies
- The Prediction of the In Vitro Release Curves for PLGA-Based Drug Delivery Systems with Neural Networks
- Promising Ursolic Acid as a Novel Antituberculosis Agent: Current Progress and Challenges
- Proof of principle concept for the analysis and functional prediction of rare genetic variants in the CYP2C19 and CYP2D6 genes
- Fabrication, characterization, and docking studies of furosemide-loaded nanosponges using the emulsion solvent diffusion method
- The Effect of Concomitant Administration of Proton Pump Inhibitors on the Pharmacokinetics of CDK4/6 Inhibitors in Rats: Implications for the Evaluation of Hepatic and Transporter-Mediated Drug–Drug Interactions
- Mining the Metabolic Capacity of Clostridium sporogenes Aided by Machine Learning
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.