StillMe: A Practical Framework for Building Transparent, Validated Retrieval-Augmented Generation Systems
Abstract
StillMe is an open-source, transparency-first framework for building validated Retrieval-AugmentedGeneration (RAG) systems designed to address three critical issues in modern AI: black-box opacity,hallucination, and knowledge cutoff limitations. Rather than modifying or retraining Large Language Models (LLMs), StillMe demonstrates thatcommercial LLMs can be transformed into ethical, auditable systems by surrounding them with atransparent architecture consisting of: • Continuous Learning: Automated ingestion from trusted sources (RSS feeds, arXiv, CrossRef, Wikipedia) every 4 hours, overcoming traditional LLM knowledge cutoff limitations. • Multi-layer Validation Chain: Six validators (citation, evidence overlap, numeric consistency, uncertainty, ethics, fallback) that enforce grounding, intellectual humility, and safety while reducing hallucinations. • Full System Transparency: Every response includes citations, validation results, and an audit trail. All learning sources, metrics, and validation outcomes are fully visible through dashboards and APIs. We evaluate StillMe on the TruthfulQA benchmark. On a 50-question subset, StillMe achieves 56%accuracy (vs. 52% for GPT-4 under identical conditions), 100% citation rate, and a 70.6% transparencyscore. On an extended 634-question evaluation, StillMe achieves 99.7% citation coverage and 70.9%transparency score, demonstrating the robustness of the validation chain across challenging subsets. Unlike traditional approaches focusing…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.