SourceMonitoring Latent World States in Language Models with Propositional Probes
In a gender-bias evaluation with 400 templated contexts specifying genders and stereotypically male or female occupations, both prompting and probing (binding similarity) show higher accuracy on pro-stereotypical than anti-stereotypical contexts. However, the probing method is significantly less biased than prompting. The authors conclude that gender bias influences the model in at least two ways: it affects how binding is done (internal world state construction) and how the model responds to queries about that state. Probing mitigates the latter but not the former.