Preventing Content Leakage in LLM-Based Medical RAG: Structure-Only Retrieval for Faithful Clinical Summarization
Abstract
Retrieval-Augmented Generation (RAG) improves factual grounding in large language models (LLMs), yet applying RAG to clinical summarization introduces content leakage, the risk that models copy patient-specific information from retrieved cases into generated summaries, a patient-privacy and clinical-safety risk. This paper introduces Structure-Only RAG, a retrieval strategy that treats retrieved clinical cases solely as structural templates while constraining all factual extraction to the source note. Each clinical section is generated independently, using only the current patient's note as input. Our two-stage pipeline combines parameter-efficient fine-tuning (QLoRA) with medical-domain retrieval using S-PubMedBERT embeddings and MedCPT cross-encoder reranking. Evaluation across six transformer architectures on MIMIC-IV discharge records shows that: (1) Structure-Only RAG reduces hallucination rates by up to 12.4 percentage points over fine-tuning alone; (2) standard RAG with full content retrieval increases hallucination by 12 percentage points, while structure-only RAG reduces it; and (3) improvement magnitude correlates inversely with baseline model faithfulness. To verify that the architectural barrier holds end-to-end on real generations, we conduct an automatic entity-level leakage audit on the full held-out test set, scanning every generated medication, diagnosis, procedure, and laboratory value for cross-patient origin. A control experiment running the identical…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.