IC-783Stable Diffusion v1.5's conditional probability pθ(x|c) is heavily biased by the unconditional probability pθ(x), making it unreliable as a condition-alignment metric
The authors measured log pθ(x|c) and log pθ(x) for 100 images generated from the prompt 'woman with green hair, wearing sunglasses and dresses' using Stable Diffusion v1.5. They found that images with higher log pθ(x|c) tend to have markedly lower CLIP scores, and that the ranking tendency of log pθ(x|c) is almost identical to that of log pθ(x). This indicates the conditional probability does not properly reflect alignment with the condition, but is dominated by the unconditional image probability, likely because user-specified conditions are often unseen in the training data.
Evidence
correlational
Key metric
Fig. 2: average CLIP score of top n% images cherry-picked by log pθ(x|c) is lower than by log pθ(x|c) − log pθ(x); the tendency of log pθ(x|c) and log pθ(x) are 'almost the same'
Caveat
Demonstrated on a single prompt and a single model (SD v1.5); the authors hypothesize the effect is general to diffusion models but do not test it on others in this preliminary experiment.