Task-Oriented RAG Evaluation: Derived Evaluation Results
Abstract
This repository contains the derived per-sample evaluation results produced from the **Task-Oriented RAG Evaluation: Raw Generation and Retrieval Results** dataset. The records combine experimental outputs with reference information from the Question-Answer-Citation (QAC) benchmark and a broad set of computed evaluation measures. These measures characterize answer quality, retrieval effectiveness, grounding, faithfulness, hallucination, attribution, refusal behavior, context usage, and system performance. The dataset includes evaluated standalone chat-mode and retrieval-augmented generation (RAG) observations across multiple model, hardware, platform, embedding, and reranking configurations. The repository also includes the persistent retrieval databases associated with the experimental RAG configurations. These consist of ChromaDB vector stores and Whoosh lexical indexes corresponding to the evaluated embedding and hardware configurations. The dataset is intended to support reproduction of the analyses reported in the associated task-oriented RAG evaluation research and independent analysis of RAG behavior by information-seeking task category. Each line in the JSONL result files represents one evaluated experimental observation. --- Relationship to the QAC Benchmark and Raw Results The derived dataset was produced from two principal research objects: 1. the Question-Answer-Citation benchmark, which supplies task categories, questions, reference answers, and authoritative…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.