Knowledge Base Scope Matters: A Comparative Study of Retrieval-Augmented Generation in Academic Library Collection Development
Abstract
Abstract As academic libraries explore large language models (LLMs) for collection development, the impact of knowledge base scope in Retrieval-Augmented Generation (RAG) remains underexamined. This study compares three configurations—NoRAG baseline, a broad geoscience knowledge base (RAG-Geo), and a specialized remote sensing subfield knowledge base (RAG-RS)—using 525 monographs in astronomy and geosciences (CLC class P). Results show that RAG-based configurations reduce score standard deviation from 21.36 (NoRAG) to 17.68 (RAG-Geo) and 18.67 (RAG-RS), while median scores drop from 75 to 70 and 65, respectively. Citation markers appear in 77.1% (RAG-Geo) and 73.5% (RAG-RS) of scoring rationales at higher score levels, indicating increased evidence grounding. The specialized knowledge base (RAG-RS) leads to score reductions of 9–11 points in subfields closely related to its focus (surveying, geophysics, atmospheric sciences), whereas the broad knowledge base (RAG-Geo) yields smaller reductions of 5–7 points and more uniform adjustments across disciplines. These findings demonstrate that knowledge base scope actively modulates RAG evaluation patterns. The study provides empirical guidance for academic libraries in selecting knowledge bases and introduces a quantitative proxy—citation proportion—for evidence grounding.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.