Text-to-image generation with diffusion models
Abstract
In recent years, text-to-image generation models have greatly improved in quality, currently achieving remarkable results in generating high-quality images. The current state of the art of image generation models is diffusion models. At their core, diffusion models generate images by iteratively removing noise from a sample drawn from a Gaussian distribution. Popular diffusion models are DALL-E 2, Glide, Imagen, and Stable Diffusion. This dissertation focuses on stable diffusion models where, through an in-depth and detailed understanding of its functioning, we aim to identify its current limitations and then implement solutions to mitigate some of these limitations. We address the limitation of generating higher-quality details and more complex structures, like hands and human faces, where we evaluated the consistency diffusion decoder used in DALL-E 3. We found that by using this model with stable diffusion, we achieve higher-quality details, especially for low-resolution images. Additionally, we addressed the limitations in image-text alignment on the stable diffusion 2 model, in which the alignment measures how well the generated image reproduces what the text prompt describes. To tackle this limitation, we trained our model with a mixture of text prompts from the LAION-400M dataset and prompts that were synthetically generated from the images of the dataset. We also upsampled these prompts using the GPT-3.5-turbo model to a longer and more descriptive form. Finally, we…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.