IC-1232GPT-2 XL and GPT-J exhibit knowledge conflict when subjected to reverse and composite knowledge edits, with ROME and MEMIT showing near-total failure on reverse edits
The paper applies pairs of logically connected edits to GPT-2 XL and GPT-J using four editing methods (FT, MEND, ROME, MEMIT). In the reverse-edit setting (editing inverse relations), ROME and MEMIT fail almost completely: their conflict scores (CS) drop to 0.00 and negative values, meaning the model retains the old fact alongside the new one. In the composite-edit setting (two edits linked by a logical rule through a tied fact), all methods cause significant damage to the tied fact, with tied-fact-damage (TFD) scores reaching 88.75 for FT on GPT-2 XL. The paper attributes ROME and MEMIT's reverse-edit failure to their poor performance on reverse relation reasoning.
The paper notes that ROME and MEMIT's failure on reverse edits may be caused by poor performance on reverse relation reasoning (citing Berglund et al., 2023), suggesting the conflict may partly reflect a pre-existing model limitation rather than a pure editing artefact.