The paper evaluates 28 LMs on sentence pairs that are gender-invariant and free of gender-correlated words, as determined by PMI statistics from the Pile corpus. Using the Unstereo Score (US) metric with epsilon=0.2175 (allowing up to 1.65x probability ratio), all models show low fairness across five benchmarks. The highest scores are 40.72% (GPT-J-6B on USE-5) and 43.92% (OPT-125M on Winogender), far from the ideal 100%. This indicates that gender bias in LMs does not solely stem from the presence of gender-related words in the input sentences.
Evidence
correlational
Key metric
US scores range from 9.04% (MPT-30B on USE-5) to 43.92% (OPT-125M on Winogender) across 28 models and 5 benchmarks at |maxPMI(s)| <= 0.65
Caveat
The evaluation focuses on binary gender (he/she) in English only; applicability to non-binary identities or other demographic groups remains open. The USE benchmarks were generated by a single model (GPT-3.5-turbo), which may introduce model-specific artifacts.