Mixture of Global and Local Experts With Diffusion Transformer for Controllable Face Generation
Abstract
Controllable face generation poses critical challenges in generative modeling due to the intricate balance required between semantic controllability and photorealism. While existing approaches struggle with disentangling semantic controls from generation pipelines, a fundamental bottleneck lies in their inability to simultaneously capture holistic facial structure and region-level semantics, resulting in coarse, entangled representations that hinder fine-grained attribute control. We revisit this challenge through the lens of expert specialization and introduce Face-MoGLE, a novel framework featuring: 1) Semantic-decoupled latent modeling through mask-conditioned space factorization, enabling precise attribute synthesis; 2) A mixture of global and local experts, motivated by the need to decouple holistic structural reasoning from region-specific semantic refinement, achieving fine-grained controllability that monolithic encoders cannot provide; 3) A diffusion-aware dynamic gating network that produces time- and spatially-dependent coefficients evolving with the denoising process. In addition, to alleviate the scarcity of evaluation benchmarks for multimodal face generation, we further extend two existing face datasets into enriched multimodal versions through a semi-automatic annotation pipeline, resulting in MM-FFHQ-Female (derived from FFHQ-Text) and MM-FairFace-HQ (derived from FairFace). These datasets provide both textual attributes and semantic masks, facilitating…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.