Analyzing Coarse-to-fine Generation of Diffusion Models through Image Editing Perspectives
Abstract
Diffusion models involve a forward noising process and a reverse denoising process. Their diffusion-like sampling process have an intrinsic nature of coarse-to-fine generation (i.e., the initial stages build coarse-grained outlines and the latter stages focus on fine details), which has been implicitly demonstrated in prior works. The goal of this work is to present an analysis method that explicitly reveals the diffusion models' coarse-to-fine behavior, through conducting image editing that mixes the latent features of a pair of samples (source and reference). Different from the prior analysis that defined a specific stage based on the step number T (e.g., fine: [0, 333], medium: [333, 666], and coarse: [666, 999]), our analysis more carefully partitions the generative sequence based on variation of noise and U-Net latent features during the sequence. We then show that, with a few modifications, our analysis method can perform the image editing task working on top of existing pre-trained diffusion models while allowing controllability, without any additional training or fine-tuning. Finally, we propose an extended metric based on LPIPS (Learned Perceptual Image Patch Similarity), which is more suitable for evaluating the controllable image editing task. Experiments on several widely-used benchmarks show the effectiveness of our method. Our code is available at: https://github.com/gyeomo/Coarse-to-fine_Generation_of_Diffusion
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.