PIPA: Prior-Driven Prompting With Diagnosis-Oriented Retrieval-Augmentation for 3-D Radiology Report Generation
Abstract
Automatic radiology report generation has gained increasing attention for its potential to assist in clinical reporting and reduce the workload of radiologists. Existing 3D radiology report generation methods employ multi-modal foundation model to encode volume-text inputs and produce diagnosis reports, while they ignore the characteristics of 3D volumes including much background regions and suffer from generating hallucinations, especially in medical domain that contains many uncommon professional terms. In this paper, we aim to efficiently adapt the pre-trained foundation model to specific 3D radiology report generation, and present a Prior-drIven Prompting with diagnosis-oriented retrieval-Augmentation (PIPA) framework. In PIPA, we design a Prior-drIven Prompting (PIP) strategy to exploit diagnostic knowledge from input volumes and a Diagnosis-oriented volume-report retrieval-augmentation Generation (DIG) module to explore beneficial knowledge from external database. Specifically, in PIP, to take full advantage of the patient's clinical information, e.g., age and symptoms, and the possible disease information, e.g., brain tumor, edema, we formulate them as the patient and disease priors to mine clinical relevant knowledge. Furthermore, we propose utilizing visual and textual embeddings as queries to retrieve similar external data by devising a diagnosis-oriented retrieval-augmentation scheme for leveraging more report resources as references for LLM to produce accuracy…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.