Zero-shot Thoracic Oncologic History Generation for Radiologists Using Retrieval-augmented Large Language Model Pipeline
Abstract
Background Gathering oncologic history is necessary but inefficient. Recent efforts to generate clinical summaries using large language models (LLMs) have relied on manual information retrieval, prompt tuning, and subjective evaluation, limiting clinical translation. Purpose To develop and evaluate a zero-shot LLM pipeline that programmatically generates structured oncologic histories using retrieval-augmented generation to ground LLM outputs. Materials and Methods Retrospective electronic health record data were collected for thoracic oncology patients at a cancer center from January 2018 to April 2024. Twenty surveyed radiologists selected 10 clinical parameters for summarization. A retrieval-augmented generation pipeline was developed to filter clinical data. Three Health Insurance Portability and Accountability Act-compliant LLMs-OpenAI's GPT-4o mini, o3-mini, and GPT-5-mini-produced structured summaries using a defined data schema and a zero-shot prompt approach. Summaries were manually evaluated against retrieved clinical text, with completeness reference standard based on the output of GPT-5-mini. Two radiology residents were timed summarizing clinical parameters, establishing a 240-second benchmark. Comparisons between models were conducted using paired t tests for accuracy, completeness, cost, and processing time. Cost-benefit analyses were conducted using manual extraction time, automated summary generation time, and national survey data on radiologist…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.