Explainability of decoder-only clinical large language models: A scoping review
Abstract
Clinical large language models (LLMs) are increasingly used for documentation, diagnosis, and decision support, but their opaque reasoning can limit clinician trust, regulatory assessment, and safe deployment. Explainability research has expanded rapidly, yet existing reviews largely address traditional machine learning or general-domain LLMs. We conducted a PRISMA-ScR scoping review to map explainability approaches for decoder-only clinical LLMs with over one billion parameters, searching PubMed, Scopus, Web of Science, ACM Digital Library, and arXiv through early 2026. Among 69 included studies, LLM-native generative and interactive methods dominated (58.0%, n = 40), spanning chain-of-thought rationales, retrieval-augmented evidence citation, and agentic decomposition. Intrinsic by-design methods accounted for 24.6% (n = 17); post-hoc XAI methods accounted for 17.4% (n = 12). General medicine was the most represented clinical domain, and diagnosis was the dominant task. Proprietary models were used in 75.4% of studies, yet every mechanistic analysis relied on open-source models, revealing a transparency asymmetry: most deployed models are the least transparent. Although 59.4% of studies quantitatively evaluated explanations, metrics remain non-standardized and rarely assess faithfulness. Local explanations predominated, and no study prospectively evaluated explanations in live clinical workflows. These findings show that clinical LLM explainability has shifted toward…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.