RAG-Governor: Unified Governance for Trustworthy and Cost-Efficient Retrieval-Augmented Generation with SLO Guarantees
Abstract
Retrieval-Augmented Generation (RAG) grounds language model outputs in external knowledge to reduce hallucinations, yet current architectures treat retrieval as a one-time preprocessing step that cannot detect errors during generation, adapt to varying source reliability, or optimize costs under latency constraints. We present RAG-Governor, a unified governance layer that reframes RAG as continuous monitoring and adaptation. The system integrates three modules: a validation engine that performs real-time entity extraction and entropy-based uncertainty monitoring to intercept hallucinations during token generation, a source manager that maintains dynamic trust scores through cross-verification feedback loops, and a hierarchical memory system combining knowledge graphs with semantic caching to reduce retrieval costs while ensuring information freshness. Evaluation across four benchmarks spanning factual and temporal question answering demonstrates that RAG-Governor reduces hallucination rates by up to 42% compared to reliability-aware baselines while decreasing retrieval costs by 40–52% through intelligent cache utilization. The system achieves 56% higher throughput than standard RAG and maintains SLO violation rates below 20% under strict latency budgets where existing methods exceed 45%. These results establish continuous governance as a practical paradigm for deploying trustworthy and cost-efficient RAG systems in production environments.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.