Perturbation-Restrained Sequential Model Editing

2025-01-22 · ICLR 2025 Poster · anchor

Findings

• IC-348Sequential parameter-modifying editing causes progressive degradation of general abilities in GPT-2 XL, Llama-2 7B, and Llama-3 8B, driven by growth in the condition number of the edited matrix
• IC-349Larger LLMs (Llama-2 7B, Llama-3 8B) suffer more severe general ability degradation than smaller models (GPT-2 XL 1.5B) under the same number of sequential edits
• IC-350Editing conceptual knowledge with rome on Llama-2 7B is harder than factual knowledge editing, with the model failing to update concept-instance relationships