Controllable Generation with Diffusion Models and Applications in Medical Imaging
Abstract
Deep generative modeling has undergone rapid advancement in recent years, with diffusion models (DMs) emerging as a dominant framework for high-fidelity and diverse image synthesis. A core strength of DMs is their inherent controllability, offering the ability to steer generation through various conditions. Despite these advantages, achieving reliable controllability remains challenging, as issues such as concept entanglement, multimodal misalignment, and limited adaptability to specialized domains continue to restrict the practical deployment of DMs. These challenges become even more pronounced in medical imaging, where controllability is essential for tasks such as targeted data augmentation, but research on controllable generation using DMs within medical imaging remains limited. In this thesis, I investigate controllable generation with DMs from both theoretical and application-oriented perspectives. First, I advance the understanding of controllability in general-purpose text-to-image DMs by introducing an attention-driven framework for disentangling and customizing multiple visual concepts from a single image. This framework effectively mitigates issues of feature fusion and asynchronous learning across concepts that degrade customization quality, improving fidelity and control in customized generation. Building on the insights into attention-based guidance, I then explore controllable generation in medical imaging, focusing primarily on dermoscopic and chest X-ray…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.