A Controlled Comparison of Sparse, Dense, and Hybrid Retrieval Strategies in Retrieval-Augmented Generation Systems
Abstract
Retrieval-Augmented Generation (RAG) improves the factual grounding of Large Language Models by retrieving external documents and using themas context during answer generation. Since the retrieved context directly shapesthe final output, the choice of retrieval strategy is a central design decision in RAG systems, yet existing studies often evaluate retrieval in isolation orvary several pipeline components at the same time, which makes it difficult to attribute observed differences to the retrieval method alone. This thesis addresses that gap through a controlled experiment comparing sparse, dense, and hybrid retrieval in a fixed RAG pipeline, in which the dataset, chunking strategy, reader model, prompt structure, and evaluation procedure are held constant while only the retrieval method varies. The retrieval methods are evaluated on the NVIDIA TechQA-RAG-Eval dataset across retrieval effectiveness, final answer quality, and computational efficiency, with both mean scores and pairwise statistical tests reported. The results show that hybrid retrieval achieves the strongest retrieval effectiveness, with statistically significant improvements over BM25 on several metrics and smaller, partially significant differences from dense retrieval. At the final answer level, hybrid retrieval produces the highest mean similarity-based scores, but most differences are not statistically significant under the evaluated sample size, and faithfulness, which is evaluated through an LLM-based…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.