A Biomedical Benchmark and Modular RAG Framework for Text2Cypher
Abstract
Abstract Background Biomedical Knowledge Graphs (bioKGs) are increasingly used to integrate heterogeneous biomedical data, including genomic interactions, disease associations, and drug–target relationships. Property graph databases, such as Neo4j, provide an effective framework for representing and querying these data through the Cypher query language. However, writing Cypher queries requires substantial technical expertise, limiting the accessibility of bioKGs to domain experts. Although Large Language Models (LLMs) have recently shown promise for translating natural language (NL) questions into Cypher queries, progress in the biomedical domain remains constrained by the limited availability of representative benchmarks and reproducible evaluation frameworks. Results We present bio2C, a FAIR-compliant biomedical benchmark comprising 2,500 curated NL-Cypher pairs covering diverse biomedical information needs, graph schemas, and progressively more expressive Cypher constructs. Building upon this resource, we introduce a modular Retrieval-Augmented Generation framework that systematically combines graph schema descriptions, retrieved NL-Cypher examples, and query execution outputs, thereby establishing reproducible baselines for future Text2Cypher research. We evaluate the resulting pipelines using both proprietary (GPT) and open-source (LLaMA) LLMs together with their fine-tuned counterparts. Experimental results show that retrieving representative NL-Cypher examples…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.