IC-082GPT-3.5, GPT-4o, Claude-3.5-Sonnet, and Llama-3.1-8B produce explanations on the BBQ social bias task that are systematically unfaithful for identity and behavior concepts while remaining faithful for context concepts, with specific patterns of hiding safety-measure influence and social bias

Report a problem with this record

Katie Matton, Robert Ness, John Guttag, Emre Kiciman

SourceWalk the Talk? Measuring the Faithfulness of Large Language Model Explanations

Using a Bayesian hierarchical model to estimate causal concept effects (CE) and explanation-implied effects (EE) via LLM-generated counterfactuals, the paper measures how well each model's natural-language explanations align with the concepts that actually drive its answers. On 30 BBQ questions, all four models show high faithfulness for context concepts (low CE, low EE) but low faithfulness for identity and behavior concepts. GPT-3.5 achieves the highest dataset-level faithfulness (f(x)=0.75), followed by Llama-3.1-8B (0.81), Claude-3.5-Sonnet (0.62), and GPT-4o (0.56). Two specific unfaithfulness patterns are identified: (1) GPT models refuse to answer when social identity information is present (a safety-alignment effect) but their explanations attribute the refusal to question ambiguity rather than to the identity information; (2) GPT models' answers shift when identity values are swapped (e.g., selecting the 'rich' person more often), yet their explanations never mention the identity concept.

Evidence
correlational
Key metric
f(x) = 0.75 (90% CI [0.42, 1.00]) for GPT-3.5; f(x) = 0.56 (CI [0.24, 0.86]) for GPT-4o; f(x) = 0.62 (CI [0.28, 0.91]) for Claude-3.5-Sonnet; f(x) = 0.81 (CI [0.49, 1.00]) for Llama-3.1-8B
Caveat
Results are based on a subsample of 30 questions due to cost constraints; the authors note results are stable for n >= 15 but may not be fully representative of the entire dataset. The method relies on GPT-4o as an auxiliary LLM for counterfactual generation, which sometimes produces errors (6% error rate on BBQ).
Model
GPT-3.5 / ChatGPT-3.5 GPT-3.5-turbo-instruct, GPT-4o GPT-4o-2024-05-13, Claude 3.5 Claude-3.5-Sonnet-20240620, Llama 3.1 8B
Concepts
Explanation faithfulness
Datasets
BBQ [eval]
Related work
Atanasova et al. 2023 [compared-to], Siegel et al. 2024 [compared-to]
Related findings
IC-083
Extraction
automatic-extraction