Abstract 4144884: Optimizing Large Language Models for Interpreting American Heart Association Dietary Guidelines in Nutrition Education for Cardiovascular Disease Prevention: A Retrieval-Augmented Generation Framework
Abstract
Introduction:
Large language models (LLMs) show proficiency in clinical capabilities such as responding to medical inquiries, and summarizing information, but their application in patient education requires evaluation. This study optimizes LLMs' interpretation of dietary guidelines for nutrition education using a framework that integrates American Heart Association (AHA) guidelines with Retrieval Augmented Generation (RAG), outperforming the baseline models.
Methods:
Analyzing 105 dietitian-authored visit notes (623,214 words) with Latent Dirichlet Allocation revealed two themes (coherence score 0.9). We posed 30 questions on these themes, along with the prompt, “Answer the following question as if you were a dietitian and cite sources from AHA,” three times to the baseline Perplexity (perplexity.ai), GPT-4o (OpenAI) models, and a customized LLM framework using RAG with formatted guidelines and citations applied to the Llama 3 70B model (Meta). Three expert reviewers evaluated the responses, scoring 0 or 1 for reliability, appropriateness, guideline adherence, and harm (Figure 1). Mean scores across groups were compared using analysis of variance.
Results:
Our LLM framework scored higher than baseline Perplexity and GPT-4o models on reliability (0.57±0.45 vs. 0.24±0.42, 0.11±0.25; p70%, p<.001) indicates high reviewer agreement.
Conclusion:
The baseline models scored lower across all measures and included Type 2 errors. While the optimized model shows promise, improvements…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.