Mass-Editing Memory in a Transformer
Abstract
Recent work has shown exciting promise in updating large language models with new memories, so as to replace obsolete information or add specialized knowledge. However, this line of work is predominantly limited to updating single associations. We develop MEMIT, a method for directly updating a language model with many memories, demonstrating experimentally that it can scale up to thousands of associations for GPT-J (6B) and GPT-NeoX (20B), exceeding prior work by orders of magnitude. Our code and data are at https://memit.baulab.info.
Read the paper
Related studies
- Diarylethenes for Memories and Switches
- Long-Term Potentiation and Memory
- Social Memory Studies: From “Collective Memory” to the Historical Sociology of Mnemonic Practices
- The Cognitive Neuroscience of Working Memory
- Memory aging and brain maintenance
- Effector and Memory CTL Differentiation
Go deeper in Evidence Hub
Evidence Hub adds a study data sheet (design, sample size, evidence level, key numbers), an AI summary and a chat grounded in the paper's text.