Prototype-guided Multimodal Retrieval for Knowledge-assisted Interventional Radiology
Abstract
Interventional radiology (IR) requires joint reasoning over procedural images and domain-specific clinical knowledge. Existing medical retrieval-augmented generation (RAG) methods are mainly text-oriented or designed for general medical vision-language tasks, and therefore remain limited in retrieving fine-grained visual-textual evidence for IR scenarios. To address this limitation, we present Prototype-guided Retrieval for Interventional Medical Assistance (PRIMA), a multimodal RAG framework that jointly leverages multimodal imaging and clinical text to support IR decision-making. PRIMA constructs a multimodal IR knowledge index through anatomy-aware visual-textual alignment and modality-preserving representation learning. It then introduces domain-informed prototype learning to organize IR concepts, enabling prototype-guided retrieval that re-ranks evidence using both query similarity and prototype affinity. We conduct comprehensive evaluations on literature-curated and clinically collected IR datasets. Experimental results show that PRIMA consistently improves generation quality, question-answering accuracy and expert-rated clinical interpretability compared with existing RAG baselines. These findings demonstrate the effectiveness of clinically grounded prototype-guided retrieval for multimodal knowledge assistance in interventional radiology. Related resources are available at https://github.com/StonHamA/PRIMA.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.