RAG Evaluation Dataset and Results
Abstract
This dataset supports the findings of the study "Knowledge Base Scope Matters: A Comparative Study of Retrieval-Augmented Generation in Academic Library Collection Development". It includes: Metadata of 525 English-language academic monographs classified under CLC class P (Astronomy and Earth Sciences), including title, author, publisher, year, CLC code, and book description. Full scoring results from three RAG configurations (NoRAG, RAG-Geo, RAG-RS), each containing a numerical quality score (0–100) and a textual rationale for each monograph. Python source code for the automated evaluation pipeline, including retrieval, scoring, statistical analysis (correlation, descriptive statistics), and visualization (boxplots, heatmaps, dual-axis plots). The knowledge base articles were sourced from the Web of Science Core Collection (SCIE). Due to licensing restrictions, only the article DOIs can be provided upon request. The Qwen3-8B model and nomic-embed-text model are open-weight and publicly accessible.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.