Evidence Hub

Incremental Image Generation with Diffusion Models by Label Embedding Initialization and Fusion

Bing Li, Dong-Dong Ren, Hao Liu, Tinghao Yu, Chenglei Peng, Yang Gao

Proceedings of the 1st on Continual Learning meets Multimodal Foundation Models: Fundamentals and Advances, 2024

Abstract

Recent diffusion models have excelled in image generation, but adapting them incrementally to new, unseen classes remains difficult. This paper presents LeifDM, an incremental diffusion model that efficiently adapts pre-trained models to new classes in data-scarce environments, minimizing catastrophic forgetting. LeifDM employs a two-stage, classifier-free framework: first, it learns new weights for unseen classes through a fusion strategy, then applies these as initial label embeddings for conditional image generation. This enables LeifDM to produce diverse and realistic images of new classes, even with limited data.

Read the paper

Related studies

Go deeper in Evidence Hub

Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.

Open this study Sign up free