Deterministic Fact Validation for Retrieval-Augmented Generation of Clinical Data Reviewer's Guides: A Proposed Architecture
Abstract
Abstract Objective: Data Reviewer's Guides (DRGs) accompanying CDISC-compliant clinical trial datasets are increasingly drafted using retrieval-augmented generation (RAG). The most detailed published account of this practice reports human review as the only safeguard between a generated draft and a submitted document, with no automated step checking generated claims against the study's own structured metadata before a reviewer opens the file. This paper proposes an architecture to close that gap.
Methods:: We propose a deterministic, non-large-language-model validation layer inserted between draft generation and human review, organized around a fact contract that converts each checkable clinical assertion, such as a subject count or treatment-arm allocation, into a claim-source-method-result tuple resolved by direct lookup against source metadata rather than by a second model's judgment. Because no such system has yet been implemented or deployed, we conducted a disclosed, parameterized simulation of 2,000 documents per condition comparing four pipeline configurations: a baseline LLM, retrieval-augmented generation (RAG), RAG with LLM-based self-verification, and RAG with the proposed deterministic validator, across eleven metrics spanning factual accuracy, validation coverage, reviewer workload, and latency.
Results:: Self-verification achieved near-complete validation coverage (1.000) but recovered only 48.0 percent of true errors among the claims it checked, with an 8.1…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.