Evaluating retrieval-augmented generation for guideline-grounded textual planning in implant dentistry: A comparative study
Abstract
OBJECTIVES: To evaluate whether a retrieval-augmented generation (RAG) framework can enhance citation precision and improve the clinical reliability of large language models (LLMs) in implant dentistry.
METHODS: A domain-specific knowledge base was constructed using major international consensus reports and established clinical treatment guides. Forty standardized clinical vignettes (35 real-world, 5 synthetic) were each processed twice by both a standard LLM (GPT-4o) and a RAG-LLM, yielding 160 treatment plans. Intra-model agreement was quantified using the weighted Cohen's kappa. Two independent experts evaluated the plans across five clinical domains using a double-blinded, consensus-driven protocol.
RESULTS: Overall clinical accuracy showed no significant difference between models (P = 0.642). However, domain-specific analysis revealed that the RAG-LLM significantly outperformed the standard model in evidence traceability and citation precision (P = 0.046). Neither model exhibited severe literature fabrication. Conversely, the standard LLM demonstrated superior biomechanical spatial planning (P = 0.025). The RAG architecture exhibited "retrieval bias" in soft-tissue complication scenarios, leading to inappropriate hard-tissue augmentation. Sensitivity analysis revealed the RAG model's superiority diminished in out-of-distribution synthetic cases lacking thematic overlap with the knowledge base.
CONCLUSIONS: Implementing RAG significantly improves evidence traceability,…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.