Large Language Models for Clinical Trial Protocol Assessments
Abstract
The purpose was to evaluate the utility of large language models (LLMs) for reviewing the statistical analysis plan (SAP) and pharmacokinetics–pharmacodynamics (PK–PD) components of clinical trial protocols. Clinical trial protocols and SAPs were obtained from clinicaltrials.gov for a testbed of 15 small‐molecule drugs, biologics, and global antibiotic and public health interventions. The GPT‐4o (ChatGPT) LLM was used to elicit study design attributes, relevant guidelines, and detailed SAP evaluations with prompts engineered to the persona of a regulatory expert. The SAP methodology was assessed against the Food and Drug Administration’s (FDA) E9 Statistical Principles for Clinical Trials guidance. The SAP evaluation outputs were assessed in post hoc analyses with ChatGPT and Grok, based on a rubric that evaluated the accuracy of primary outcome identification, the correctness of statistical methodology, compliance with the FDA E9 guidance, and clinical interpretability. PK–PD analysis plans were assessed on the accuracy of PK–PD objectives and measures and PK analysis methods. ChatGPT accurately identified the disease, intervention, and comparator groups for all trials, as well as the study sample size for 14 out of 15 trials. The most frequently cited guidelines were the FDA’s E9 guidance for SAP and the FDA Guidance for Industry: Population Pharmacokinetics for PK–PD. ChatGPT outputs of the SAP and PK–PD analysis plans were clear and organized, demonstrating a…
Read the paper
Related studies
- Improving sleep quality leads to better mental health: A meta-analysis of randomised controlled trials
- Tocilizumab in patients with severe COVID-19: a retrospective cohort study
- A new aging measure captures morbidity and mortality risk across diverse subpopulations from NHANES IV: A cohort study
- Fruit, vegetable, and legume intake, and cardiovascular disease and deaths in 18 countries (PURE): a prospective cohort study
- Are noise and air pollution related to the incidence of dementia? A cohort study in London, England
- Long COVID symptoms and duration in SARS-CoV-2 positive children — a nationwide cohort study
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.