Context Engineering for Local Retrieval-Augmented Generation
Abstract
Local Retrieval-Augmented Generation can change behavior under tiny prompt edits or context reorderings, yet practical testing often uses only one query formulation. This report presents a local context-engineering framework for perturbation robustness, reproducibility, and budget-aware evaluation. The system logs each run as a structured capsule, computes lexical, retrieval, fluency, and semantic metrics, and uses a calibrated decision-gate to skip, reduce, or execute the perturbation suite. The augmented capsule- derived dataset has 3,570 perturbation rows. The retraining was done on 2,619 rows from the augmented data with enriched observed-break labels; 446 of those rows had BLEU scores, 1,668 had perplexity values, and 1,668 had semantic-judge output values. The trained gate achieved 0.915 ROC-AUC, 0.891 PR-AUC, 0.913 accuracy, 0.863 F1 score, and 76.5% expected cost savings on the held-out augmented test split.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.