Development and Evaluation of HEAL A Bilingual English-Luganda RAG Chatbot for Disease Surveillance in Uganda
Abstract
Abstract Frontline health workers in Uganda are often unable to use disease surveillance guidelines effectively: the Pdf documents are long, technical, and written in English, leaving Luganda-speaking workers without practical access. Off-the-shelf large language models generate plausible but ungrounded advice that can contradict national protocols. We developed HEAL, a bilingual chatbot using retrieval-augmented generation to deliver Uganda's Integrated Disease Surveillance and Response guidelines in English and Luganda, built on GPT-4-turbo with a fine-tuned MBart translation model trained on parallel English-Luganda sentence pairs. Six disease surveillance experts rated HEAL against three commercial models on field-generated questions across relevance, coverage, and coherence, while twenty frontline health workers gave qualitative feedback. HEAL scored higher than all baseline models in pooled analysis, with reviewers rating it highest most consistently. Grounding a large language model in national guidelines alongside a local-language translation layer produced higher expert ratings than commercial alternatives, an approach adaptable to other languages and guidelines.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.