Design and Evaluation of a Retrieval-Augmented Generation System for Czech
Abstract
The goal of this thesis is to design, implement, and evaluate a Retrieval-Augmented Generation (RAG) pipeline for Czech. We systematically analyze individual RAG compo- nents, including PDF preprocessing, text chunking, and both embedding and generative models. We evaluate a BM25 baseline and eleven embedding models using two Czech datasets: DaReCzech-Test and a custom-made Wikipedia-NLP. Additionally, we compare multiple generative models within an end-to-end RAG pipeline in a small-scale study with six participants. In retrieval, Qwen3-embedding:4B achieves the best performance. BM25 remains a strong CPU-only baseline outperforming smaller embedding models. For gen- eration, Llama 3.3 is preferred by users, whereas GPT-OSS achieves the highest factual accuracy. We release a pre-packaged Docker container based on OpenWebUI with our best-performing RAG configuration.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.