Development and evaluation of Retrieval-Augmented Generation methods for document search and question-answering
Abstract
This study investigates the development and evaluation of document-based question-answering systems (RAG, Retrieval-Augmented Generation). The aim is to create a modular base system and analyze its main components. This approach guides the tool's design so users can retrieve information from and interact with document collections using natural language queries. As the development of large language models (LLMs) continues, it is increasingly important to understand how to integrate the retrieval and creation components effectively to develop accurate, responsive, and multilingual information systems. The system is built around two main pipelines: processing documents and answering questions. The documents are cleaned by including only the textual part, split, embedded, and stored in a vector database. The retrieval component manages user queries: it identifies the relevant stored text chunks, combines them with the query, and forwards it to the language model to answer. The system is built with open-source tools and evaluated on multilingual datasets to evaluate its performance under realistic conditions. The evaluation tested different document splitting methods, embedding, and language models. The results show that retrieval quality is a key factor in overall system performance, as the retrieved text chunks directly influence the answer generation phase. Semantic chunking combined with sentence-level document splitting balanced retrieval accuracy and processing efficiency,…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.