Improving Controllability in Diffusion-Based Image Inpainting through Structured Workflows and Preference-Based Model Adaptation
Abstract
This thesis presents two complementary approaches for improving controllability in diffusion-based image generation. The first addresses limitations of existing inpainting-based virtual staging methods, which can produce objects that are semantically inappropriate, visually unconvincing, or poorly matched to the surrounding environment. To address this, an end-to-end prompt-based contextual virtual staging framework is developed that fine-tunes a diffusion model for controlled object generation and integrates it into an agentic pipeline for context-aware scene inpainting. Building on this, the second approach addresses the limitation that workflow-level control alone does not ensure that the underlying inpainting model fully internalizes prompt and scene context. To address this, a pretrained diffusion inpainting model is directly fine-tuned using preference-based optimization, together with a composite reward model and a region-aware evaluation system. Together, these approaches improve controllability at both the workflow level and the model level.
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.