An Effective and Finetuning-free Pipeline for Face Swapping with Diffusion models
Abstract
In current face-swapping tasks, a lot of work mainly relies on Generative adversarial networks (GAN) frameworks; although there are positive results in face-swapping tasks, output images still remain with drawbacks, often with noisy artifacts, particularly in scenarios involving diverse pose variation, lightning conditions, and face occlusion. On the other hand, recent work has emerged on pretrained diffusion models for their exceptional generation performance. Nevertheless, training these models is not trivial, it requires intensive resource, and the results have yet to demonstrate satisfactory efficacy. That is, training a Text-to-Image (T2I) model requires a massive dataset with high computational resources. Meanwhile, finetuning these models can lead to catastrophic forgetting due to not having enough training epochs or the volume of datasets. To overcome this disadvantage, I introduce an efficient and free-tuning pipeline designed for accurate and photorealistic facial editing leveraging pre-trained text-to-image (T2I) models. From Source images and Target images, the goal is to preserve and transfer the identity attributes of the source within the target representation, while maintaining high fidelity and preserving ID-unrelated attributes. I assume that the face-swapping’s outputs should efficiently preserve structural attributes from target images and identities of the source images. Base on these insights, our contributions specifically include: 1. I re-define the…
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.