Inference-Time Intervention / ITI / Neuron intervention (pinning activation)

anchor

Findings

• IC-1318Individual space and time neurons in Llama-2-7B causally contribute to spatial and temporal predictions [primary]
• IC-338Factuality enhancement methods (DoLa, ICD, ITI, TruthX, CD) cause large and consistent declines in context-faithfulness of LLaMA2-7B-Chat and LLaMA2-13B-Chat [builds-on]
• IC-338Factuality enhancement methods (DoLa, ICD, ITI, TruthX, CD) cause large and consistent declines in context-faithfulness of LLaMA2-7B-Chat and LLaMA2-13B-Chat [primary]
• IC-339Factuality enhancement methods produce inconsistent and modest improvements in factual accuracy on LLaMA2-Chat, with some metrics declining below baseline [builds-on]
• IC-339Factuality enhancement methods produce inconsistent and modest improvements in factual accuracy on LLaMA2-Chat, with some metrics declining below baseline [primary]
• IC-503Adding probe regression coefficients to attention head activations steers Llama-2-7B-Chat, Mistral-7B-Instruct-v0.1, and Vicuna-7B-v1.5 toward more liberal or conservative generated text [primary]
• IC-592The log-likelihood layer in LLaMA-2-7B, LLaMA-2-7B-Chat, Vicuna-7B, and Mistral-7B-Instruct produces factually incorrect answers on TruthfulQA MC1 (817 samples) due to a misalignment between the output distribution and internal attention head representations, with LM-to-head-norm accuracy gaps of 24.23 to 40.68 points. [compared-to]