A Survey on Poisoning Attacks, Defenses, andProvable Defense in the Era of LLMs
Abstract
Abstract Large Language Models (LLMs) are increasingly deployed as compound Artificial Intelligence (AI) systems that integrate retrieval, external tools, memory modules, and autonomous agents, substantially expanding both their capabilities and attack surfaces. Among emerging security threats, poisoning attacks have attracted significant attention due to their ability to manipulate model behavior by corrupting training data, manipulating adaptation pipelines, or leveraging external contextual information. This survey systematically reviews poisoning attacks and corresponding defenses in the LLMs era. It proposes a comprehensive taxonomy that organizes existing research into two primary categories: 1) weight poisoning, which manipulates the model parameter during the development pipeline, including pre-training, fine-tuning, adaptation, and pre-deployment stages; 2) context poisoning, which exploits information provided during inference through in-context learning, retrieval-augmented generation (RAG) systems, and agent flow. Based on this taxonomy, we provide a comparative analysis of representative attack techniques, highlighting their threat models, attack objectives, effectiveness, and practical limitations. Furthermore, we survey existing defense strategies, ranging from empirical defenses to emerging provable defenses, and discuss their applicability across different poisoning scenarios. Finally, we present the evolution from isolated poisoning attacks to compound…
Read the paper
Related studies
- Knowledge Management Perspective of Generative Artificial Intelligence
- Navigating Generative Artificial Intelligence Promises and Perils for Knowledge and Creative Work
- Characterizing the evolving research on enterprise content management
- Large language models and multimodal foundation models for precision oncology
- From Generative AI to Extended Reality: Multidisciplinary Perspectives on the Challenges, Opportunities, and Future of Educational Computing
- Generative Artificial Intelligence: A New Engine for Advancing Environmental Science and Engineering
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.