Enhancing image-based virtual try-on with Multi-Controlled Diffusion Models
Abstract
Image-based virtual try-on technology digitally overlays clothing onto images of individuals, enabling users to preview how garments fit without physical trial, thus enhancing the online shopping experience. While current diffusion-based virtual try-on networks produce high-quality results, they struggle to accurately render garments with textual designs such as logos or prints which are widely prevalent in the real world, often carrying significant brand and cultural identities. To address this challenge, we introduce the Multi-Controlled Diffusion Models for Image-based Virtual Try-On (MCDM-VTON), a novel approach that synergistically incorporates global image features and local textual features extracted from garments to control the generation process. Specifically, we innovatively introduce an Optical Character Recognition (OCR) model to extract the text-style textures from clothing, utilizing the information gathered as text features. These features, in conjunction with the inherent global image features through a multimodal feature fusion module based on cross-attention, jointly control the denoising process of the diffusion models. Moreover, by extracting text information from both the generated virtual try-on results and the original garment images with the OCR model, we have devised a new content-style loss to supervise the training of diffusion models, thereby reinforcing the generation effect of text-style textures. Extensive experiments demonstrate that MCDM-VTON…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.