Reward-Guided Flow Matching with k-Step Rollout for Text-to-Image Generation
Abstract
Text-to-image diffusion models can generate high-quality images from natural language prompts, but their standard training objectives do not directly optimise human preference, prompt accuracy, or visual quality. Reinforcement-learning methods such as DDPO and DPOK use external reward signals for alignment, but often rely on rewards from completed generations, which can make optimisation costly and difficult to stabilise. In parallel, flow matching and Diff2Flow improve generation efficiency through flow-based sampling, but are not primarily designed for preference alignment.This thesis investigates how preference feedback can be integrated into a Diff2Flowbased flow matching framework for text-to-image generation. The central contribution is a differentiable k-step Euler rollout that connects image-level preference feedback to the learned Diff2Flow velocity field. Starting from a noise latent, the rollout follows the predicted velocity field to obtain a generated latent, which is decoded into an image and evaluated using ImageReward. Because the reward path remains differentiable, the resulting reward loss provides a training signal to the velocity model through the rollout, while the flow matching loss retains the original data-fitting objective.Experiments further study the effect of 4-, 10-, and 20-step rollouts on alignment and generation quality. The results show that reward-guided Diff2Flow improves ImageReward-based alignment over the evaluated baselines while…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.