Optimisation of Local Retrieval-Augmented Generation (RAG) Systems on Consumer-Grade Hardware
Abstract
Managing confidential data in high-security sectors creates a critical conflict between privacy and productivity: organizations must either risk data sovereignty using cloud-based Large Language Models (LLMs) or accept inefficient, manual workflows. While local, offline Retrieval-Augmented Generation (RAG) pipelines offer a secure alternative, deploying them on consumer-grade hardware introduces severe performance bottlenecks during document ingestion, retrieval and generation. Furthermore, the lack of standardized benchmarks to quantify trade-offs between speed and accuracy leaves developers without clear architectural guidance.To address this gap, this study systematically evaluates local RAG performance under the hardware constraints using a quantitative and qualitative experimental framework. The research utilises a 2x4x2x3 factorial design comprising 48 unique experimental conditions. It cross-evaluates two different document structures — a 144-page hierarchical legal document and a 25-page tabular financial report — across four document loaders (PyMuPDF4LLM, openDataLoader, PDFPlumberLoader and UnstructuredPDFLoader), two open-source embedding models (Nomic-Embed-Text and BGE-M3) and three chunking strategies (Fixed-Size, Recursive and Semantic). Processing speeds are captured across separate pipeline stages, while informational accuracy and retrieval quality are assessed using the automated RAGAS framework.The experimental results reveal that document structure…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.