Learning-to-Rank Meets Evidence Set Selection: An Empirical Evaluation of RL Estimators for Retrieval-Augmented Generation
Abstract
Retrieval-augmented generation systems typically retrieve a candidate pool of passages and then select a small evidence set for answer generation. Standard retrieval pipelines score passages individually, often ignoring interactions among selected items. Yet evidence quality is fundamentally a set-level property, since the value of one passage depends on which others are included. We study whether stochastic ranking methods from reinforcement learning and Learning-to-Rank can be used effectively in this setting. Focusing on Plackett--Luce ranking policies, we compare naive REINFORCE, PG-Rank, and PL-Rank for evidence selection. We evaluate selectors with LLM-based set utility and held-out downstream question-answering (QA) to test whether estimator advantages from decomposable ranking survive when reward is defined on selected evidence sets rather than rank-weighted relevance. We find that PG-Rank transfers well both to document ranking with set-level reward and to setwise selection over candidate sets, while PL-Rank's structural variance and efficiency advantages weaken when set utility cannot be factored across ranked positions. We also find that learned selectors improve over the BM25 baseline in QA, though the highest-utility methods do not always achieve the strongest QA scores.
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.