anchor
Findings
- IC-116In LLaVA-7B, multi-head attention modules drive hallucination more than MLP modules, and targeted intervention on specific hallucination heads reduces the hallucination rate by up to 1.7x [compared-to]
- IC-338Factuality enhancement methods (DoLa, ICD, ITI, TruthX, CD) cause large and consistent declines in context-faithfulness of LLaMA2-7B-Chat and LLaMA2-13B-Chat [builds-on]
- IC-338Factuality enhancement methods (DoLa, ICD, ITI, TruthX, CD) cause large and consistent declines in context-faithfulness of LLaMA2-7B-Chat and LLaMA2-13B-Chat [primary]
- IC-339Factuality enhancement methods produce inconsistent and modest improvements in factual accuracy on LLaMA2-Chat, with some metrics declining below baseline [builds-on]
- IC-339Factuality enhancement methods produce inconsistent and modest improvements in factual accuracy on LLaMA2-Chat, with some metrics declining below baseline [primary]
- IC-592The log-likelihood layer in LLaMA-2-7B, LLaMA-2-7B-Chat, Vicuna-7B, and Mistral-7B-Instruct produces factually incorrect answers on TruthfulQA MC1 (817 samples) due to a misalignment between the output distribution and internal attention head representations, with LM-to-head-norm accuracy gaps of 24.23 to 40.68 points. [compared-to]