Designing and Evaluating a Retrieval-Augmented Generation (RAG) System for AI Experts
Abstract
Recent progress in generative artificial intelligence, particularly in large language models (LLMs), has enabled significant advances in natural language processing across many application areas. In expert-oriented systems, however, these models continue to exhibit critical limitations, including hallucinated content, factual inaccuracies, and reduced robustness when applied to specialized or technical domains. Retrieval-Augmented Generation (RAG) has emerged as a promising approach to mitigate these issues by explicitly grounding model outputs in external knowledge sources. This thesis examines the design and empirical evaluation of a modular RAG pipeline intended for expert applications such as technical documentation assistance and process automation. The work specifically investigates whether combining multiple retrieval modalities ranging from dense vector-based retrieval to graph-based symbolic reasoning can improve the factual grounding, relevance, and explainability of generated responses. Particular attention is paid to how different retrieval strategies affect downstream generation quality in real-world technical settings. The architecture integrates interchangeable retrieval components, context augmentation strategies, and transformer-based language models, enabling systematic comparison across multiple retrieval configurations. An iterative design and evaluation methodology is adopted, encompassing architectural analysis, implementation of retrieval and…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.