RAG vs. No-RAG benchmark dataset: LLM-judged clinical reasoning in transplant infectious disease
Abstract
Dataset supporting the study "Retrieval-Augmented Generation Reduces Synthesis-Intensive Clinical Reasoning Quality in Transplant Infectious Disease." Contains three files: all_models_rag_non_rag_judged.csv — Primary dataset. 4,911 LLM-judge records from 1,637 unique model outputs (39 models × 21 transplant infectious disease clinical vignettes × 2 conditions [RAG / No-RAG], scored by 3 independent LLM judges on 6 clinical reasoning dimensions). Includes generation-time grounding metrics and major-error annotations. expert_evaluations_linked_long.csv — Blinded domain-expert evaluations. 60 ratings from 3 experts across 10 paired RAG/No-RAG cases (accuracy, usefulness, preference). retrieval_quality_classification.csv — Per-case retrieval quality audit. Document-level classification of all 21 cases into 6 retrieval quality categories with similarity scores and per-case RAG-benefit deltas. Analysis code and figure generation scripts: https://github.com/KeatingLabNYU/rag-txid-benchmark
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.