Backdoor Attacks on Text-to-Image Diffusion Models via LoRA Plugin
Abstract
Text-to-image (T2I) diffusion models widely adopt Low-Rank Adaptation (LoRA) for efficient fine-tuning, but the “plug-and-play” nature of third-party LoRA plugins introduces potential security risks. This paper presents a malicious LoRA backdoor attack framework targeting T2I diffusion models. By freezing the base model weights and fine-tuning only the LoRA parameters, we perform minimal data poisoning (5%) during training. The model retains its original style generation capabilities under normal inputs, and generates attacker-specified target images when a specific trigger—such as an invisible zero-width character—is present. The attack achieves high stealthiness by embedding backdoors exclusively in low-rank adapter weights without modifying base model parameters. Experiments on Stable Diffusion v1.5 demonstrate that our method achieves an attack success rate (ASR) exceeding 90% while preserving the original model utility (FID increase <2%). These results expose critical vulnerabilities in third-party LoRA distribution networks and underscore the need for security vetting in open-source model sharing platforms.
Read the paper
Related studies
- Learning from First-Generation Qualitative Approaches in the IS Discipline: An Evolutionary View and Some Implications for Authors and Evaluators (PART 1/2)
- Commentary on the clinical and preclinical dosage limits of interstitially administered magnetic fluids for therapeutic hyperthermia based on current practice and efficacy models
- Prosthesis: New Technological Opportunities and Innovative Biomedical Devices
- The history and future of neural modeling for cochlear implants
- Ryanodine receptors: waking up from refractoriness
- Novel Transgenic Rabbit Model Sheds Light on the Puzzling Role of Matrix Metalloproteinase-12 in Atherosclerosis
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.