Unifying Generative Self-Supervised Paradigms with Diffusion Models
Abstract
Learning semantic representations with generative self-supervised pre-training has demonstrated immense potential in the field of Natural Language Processing (NLP). However, this success has not been fully replicated in computer vision, primarily due to two key challenges: (1) visual generation tasks are difficult to formalize into a unified task, and (2) different downstream tasks require visual representations at varying semantic levels. To address these challenges, we propose a novel Generative framework with a UNified Self-supervised training paradigm (GUNS) to learn semantic information at different granularities and unify multiple image generation tasks within a denoising diffusion model. Specifically, GUNS employs various data augmentations to construct training objectives at different visual levels and unifies these training objectives into a generative paradigm. To facilitate this multi-task pre-training, the denoising diffusion model is introduced into GUNS as the image decoder, thereby making GUNS also an image-conditional generative model. Unlike other visual self-supervised methods that can only be applied to downstream tasks via transfer learning, GUNS can be directly applied to image generation tasks such as colorization and out-painting. We evaluate GUNS on downstream recognition tasks and image generation tasks, and experimental results demonstrate that GUNS can achieve competitive performance on both tasks simultaneously.
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.