Decomposable Image Diffusion for Accurate Semantic Translation
Abstract
Recent advances in text-to-image diffusion models have achieved remarkable success in generating high-quality and realistic images from given prompt. However, these methods face challenges in handling complex prompts and ensuring faithful alignment between the generated image and the textual described in the prompts. In this paper, we propose a two-stage image-text realignment method, called Decomposable Image Diffusion, to improve the alignment between prompt and image in text-to-image generation diffusion models. Decomposable Diffusion contains two stages. Stage 1: semantic decomposition and compositional image generation, integrates linguistic structures into the diffusion guidance process by decomposing complex textual inputs into multiple semantic concepts, which are then used to guide the image generation process. Stage 2: Semantic realignment aims to realign the image with the prompt. We propose global reward and local reward to evaluate the consistency between the generated image and the prompt from both global and local perspectives. Then, we compute the global realignment loss and local realignment loss, and fine-tune the pretrained diffusion model from the first stage to achieve the realignment between the image and the prompt. Experimental results from MS-COCO, ABC-6k, and CC-500 benchmarks show that the proposed Decomposable Diffusion significantly outperforms other baseline realignment techniques in terms of visual quality and semantic similarity with the input…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.