Design and Evaluation of a RAG-Based Educational Assistant Grounded in Course Materials: A Case Study in Vocational Training
Abstract
This study presents the design, implementation, and exploratory classroom evaluation of a web-based educational assistant built on Retrieval-Augmented Generation (RAG) and Large Language Models (LLMs) for Vocational Education and Training (VET). The platform was designed to generate responses based on teacher-provided course materials, preserve source traceability, and return an abstention message when the retrieved evidence is insufficient. The assistant was deployed in an authentic classroom setting within the Higher Vocational Training programme in Network and Information Systems Administration (ASIR). Nineteen students and one instructor used the system during a practical session and completed a post-session questionnaire combining Likert-scale items with open-ended questions. The findings indicate positive student perceptions of usability, response clarity, perceived reliability, and learning support. Participants particularly valued the ability to obtain focused answers aligned with the instructional materials. The evaluation also revealed a relevant trade-off: restricting the assistant to a controlled corpus reinforced curricular consistency and perceived trustworthiness but limited its capacity to address questions insufficiently covered by the available resources. The absence of conversational memory emerged as the most frequently requested improvement. These preliminary findings suggest that course-constrained RAG assistants may constitute valuable complementary…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.