Modelpedia
Work in progress
About
Findings
Models
Concepts
Methods
Datasets
Sources
Light
Dark
Debiasing Algorithm through Model Adaptation
2024-01-16
· ICLR 2024 poster ·
anchor
Findings
IC-1078
LLaMA models (7B through 65B) exhibit gender bias in language generation, coreference resolution, and sentence likelihood, with stereotypical associations driving predictions
IC-1079
Causal tracing reveals that mid-upper MLP layers (layers 18–25 in 7B) are the primary mediators of stereotypical gender bias in LLaMA, while the last layers show negative coefficients that counter the bias