Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage
Abstract
Retrieval-augmented generation (RAG) systems combine document retrieval with a generative model to address complex information-seeking tasks like report generation. While the relationship between retrieval information coverage and generation effectiveness seems intuitive, it has not been systematically studied. We investigate the relationship between upstream retrieval metrics (coverage-based and relevance-based) and the downstream generated response's information coverage. Through experiments across two text RAG benchmarks (TREC NeuCLIR 2024 and TREC RAG 2024) and one multimodal benchmark (WikiVideo), we analyze 15 text retrieval stacks and 10 multimodal retrieval stacks across four RAG pipelines and two evaluation frameworks (Auto-ARGUE and MiRAGE). Our findings demonstrate consistent positive correlations between coverage-based retrieval metrics and information coverage in generated responses at both topic and system levels. This relationship holds most strongly when retrieval and generation objectives are aligned, and weakens as RAG pipelines become more iterative and complex. These findings provide empirical support for using coverage-based retrieval metrics as efficient proxies for RAG information coverage evaluation, reducing the need for costly end-to-end evaluation.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.