Improving Semantic Precision in Text-to-Image Diffusion Models via Latent-Space Optimization and Semantically-Parsed Evaluation
Abstract
Text-to-image diffusion models can produce visually impressive images from natural-language prompts, but they often fail to satisfy the detailed semantic constraints expressed in compositional prompts. Typical failure modes include omitted objects, merged entities, incorrect quantities, incorrect attribute binding, and leakage of one entity's attributes onto another. This thesis studies the problem of semantic precision in text-to-image generation: how faithfully a generated image satisfies the structured meaning of its prompt. The thesis makes two linked contributions. First, it presents a training-free inference-time refinement method for diffusion-based image generation. The method operates directly in latent space during denoising and uses noun-phrase-aware cross-attention objectives to improve object presence, attribute binding, and spatial separation between semantically distinct prompt entities. The approach augments an Attend-and-Excite style attention-activation objective with additional losses for attribute alignment, noun-phrase separation, and centroid separation, without retraining the underlying diffusion model. Second, the thesis introduces a structured evaluation framework for prompt-image semantic alignment. Rather than assigning a single global similarity score to the whole prompt-image pair, the framework decomposes the prompt into explicit semantic constraints, grounds relevant image regions, and verifies each constraint through targeted yes/no visual…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.