Bio-KGR: Automated Evidence-Grounded Biomedical QA Benchmark Generation for LLM Reasoning Evaluation
Abstract
Bio-KGR Benchmark v1.0 Bio-KGR is an automatically generated, evidence-grounded biomedical question-answering benchmark for evaluating the domain-specific reasoning abilities of large language models (LLMs). The benchmark combines knowledge-graph-derived reasoning structures with supporting evidence retrieved from PubMed-indexed biomedical literature. Contents:- high-quality open-ended QA pairs (SynLethKG-based, after filtering)- Reasoning structure annotations (One-hop, Two-hop, Intersection, Attribute)- PubMed evidence references (PMID) for each QA pair- Generation prompts and evaluation protocols Use Cases:- LLM biomedical reasoning evaluation- Multi-hop QA benchmark- Knowledge-grounded generation evaluation- Retrieval-augmented QA system development License: CC-BY 4.0
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.