SourceThe Hidden Language of Diffusion Models
Conceptor reveals that SD's internal representations of profession and role concepts include biased demographic associations: 'secretary' decomposes into 'womens, girl, ladies'; 'opera singer' into 'obese, overweight, fat'; 'pastor' into 'nigerian, gospel'; 'journalist' into 'refugee, jews'; 'drinking' into 'millennials, blonde'; 'professor' into 'men'; 'nurse' into 'woman, women, wife, mother'. The authors note these biases are not necessarily observable by simply looking at generated images. They demonstrate that reducing the coefficients of biased tokens while preserving other features can produce debiased generations on the same seeds.