Moral Reasoning RAG Evaluation Study: Data, Reproducibility Materials, and Statistical Analysis
Abstract
This archive contains the data, source records, analysis code, verification materials, and supplementary documentation for a controlled evaluation of a Moral Reasoning Retrieval-Augmented Generation (RAG) intervention. The study compared the same underlying language model under two conditions: an unaugmented Raw API condition and a Moral Reasoning RAG condition. The evaluation dataset contained 300 moral-reasoning questions drawn from established benchmarks, published benchmark frameworks, and researcher-created reliability scenarios. Each matched Raw API and RAG response pair was evaluated under blinded A/B presentation by three AI evaluators, producing 900 judge–question evaluation records and 1,800 individually scored responses. The archive includes: the complete 300-question provenance record and study condition key; original evaluator-result packages; batch-load and model-export records; blinded and decoded canonical analytic datasets; data-cleaning and reconstruction documentation; the Universal AI Moral Reasoning Grading Protocol; executable Python statistical-analysis code; machine-readable statistical outputs; verification records and reproduced figures; exploratory response-length and sensitivity analyses; a cluster-robust sensitivity analysis accounting for the 12 five-stage Multi-step Moral Dilemma sequences represented among the 60 MMD items; and a blind-review recalculation package designed to permit independent statistical verification of the reported results.…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.