Pengembangan Chatbot Berbasis Retrieval-Augmented Generation (RAG) Pada Domain Akademik
Abstract
Conventional academic information services often face constraints such as limited operational hours and slow response times. This research aims to develop a chatbot based on Retrieval-Augmented Generation (RAG) integrated with WhatsApp, providing an adaptive, responsive, and factual academic information service at the Informatics Department, University of Muhammadiyah Malang. The system is built using the low-code n8n platform, utilizing the OpenAI text-embedding-3-small model to vectorize knowledge base documents into a Supabase vector database, and Google Gemini as the Large Language Model (LLM) for response generation. Integrated documents comprise both textual data (academic guidelines) and structured tabular data (course schedules and faculty directories). System quality was evaluated objectively across 49 question scenarios validated by 5 active students. The functional testing using a Confusion Matrix demonstrated robust and reliable performance, achieving an accuracy of 91.8%, precision of 95.1%, recall of 95.1%, and an F1-Score of 95.1%. The implementation of the RAG method equipped with strict system prompt instructions proved effective in minimizing information hallucination by consistently triggering safe fallback rejections when the required data was absent. In terms of infrastructure, the system demonstrated high operational efficiency, with an average input consumption of 1,500–2,500 tokens, output of 100–300 tokens, and a highly economical estimated…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.