CLEAR-RAG: A Practical Framework for Evaluating Production-Grade Retrieval-Augmented Generation Systems
Abstract
Retrieval-Augmented Generation (RAG) is increasingly deployed in enterprise knowledge management, legal research, healthcare information support, and customer service. Yet production RAG systems are still routinely evaluated using single, narrow metrics that do not reflect the combined requirements of real-world deployment: retrieving the right evidence, citing sources accurately, generating answers that are faithful to that evidence, and operating within acceptable latency and cost budgets. This paper introduces CLEAR-RAG, a five-dimensional evaluation framework for production-grade RAG systems spanning Citation Quality (C), Latency and Cost (L), Evidence Faithfulness (E), Answer Relevance (A), and Retrieval Quality (R). The framework provides formal metric definitions, a 1–5 scoring rubric with domain-specific weighting presets, and an LLM-as-judge implementation that produces structured JSON scores. We describe a full reference implementation comprising both keyword and dense vector retrieval modes, a hybrid human-and-automated review workflow, and an error taxonomy covering nine failure categories. A controlled demonstration on 15 enterprise knowledge-base questions across two retrieval modes shows that CLEAR-RAG surfaces tradeoffs that are invisible to single-metric evaluation: keyword retrieval reaches the maximum Latency score (5.0) but trails on Retrieval Quality (2.8 average), while dense vector retrieval lifts Retrieval Quality to 3.7 at a modest latency cost…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.