Explainable Agentic Artificial Intelligence in Healthcare: A Scoping Review
Abstract
Background: Agentic artificial intelligence (AI) systems, characterized by autonomous goal-directed behavior, multi-step reasoning, task decomposition, and tool use, are increasingly proposed for healthcare applications. However, their autonomy raises concerns regarding transparency, accountability, and human oversight. While explainable AI (XAI) has been widely studied in traditional predictive models, less is known about how explainability is implemented within agentic architectures.
Objective: To map the emerging literature on explainable agentic AI (XAAI) in healthcare and characterize the types, scope, and forms of explainability used in these systems.
Methods: A scoping review was conducted following PRISMA-ScR guidelines. PubMed, Embase, IEEE Xplore, and ACM Digital Library were searched through November 2025. Eligible studies described healthcare-related agentic AI systems incorporating explicit explainability mechanisms. Data were extracted on system architecture, explainability type (intrinsic, post hoc, hybrid), explanation scope (local, global), explanation form, and reported clinical outcomes.
Results: Nine studies met the inclusion criteria. All systems demonstrated core agentic features, including autonomy, task decomposition, and tool integration, often within multi-agent frameworks. Explainability was predominantly intrinsic and workflow-native, typically delivered through textual reasoning traces and example-based grounding in retrieved clinical evidence.…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.