Retrieval-Augmented Generation for Reducing Hallucinations in Drug-Related Question Answering Systems
Abstract
Abstract Large Language Models (LLMs) deployed in pharmaceutical question-answering applications frequently generate clinically dangerous hallucinations — factually incorrect outputs that pose serious patient-safety risks. We propose and evaluate a Retrieval-Augmented Generation (RAG) pipeline that grounds LLM responses in a curated, citation-verified pharmaceutical knowledge base. The system combines dense passage retrieval using Sentence-BERT embeddings, FAISS vector indexing, cross-encoder re-ranking, and Llama 3.2 generation. Evaluated on a purpose-built dataset of 150 drug-related question–answer pairs spanning ten commonly used medications and multiple information categories, our RAG system reduces the hallucination rate from 47.8% to 12.3% — a 74.3% relative reduction — while improving Faithfulness from 0.52 to 0.87 and Factual Accuracy from 0.54 to 0.89. A clinical case study on warfarin–ibuprofen interaction demonstrates that RAG responses provide verifiable, citation-grounded information where the baseline produces potentially life-threatening misinformation. These results establish retrieval-augmented architectures as the preferred approach for safety-critical pharmaceutical AI and provide an open, reproducible evaluation framework.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.