Feature interaction

An explanation assigns importance to combinations of features, not only to single ones. Necessary wherever the output depends on a joint effect, as in the similarity between an image region and a word, which no single-token score can express.

Findings

• FX-001Pairwise Banzhaf interactions explain CLIP similarity more faithfully than single-score methods
• FX-003FIXLIP's strongest interaction in one CLIP example links doll to an image patch reading dollar
• IC-052GPT-2 small's IOI circuit activations are linearly decomposable into features for the io, s, and pos attributes, with the l10h0 name mover's attention decomposing into sparse pairwise feature interactions
• IC-663OPT-1.3B, LLaMA-7B, and Aquila-7B encode sparse Harsanyi interactions, with only 29-51 salient interactions out of 1024 possible on SQuAD sentences
• SY-003Sybil processes pulmonary nodules almost additively, with limited pairwise interactions