SourceInterpretable Diffusion via Information Decomposition
The paper performs prompt interventions (word omission and word swap) on real images using Stable Diffusion v2.1 and measures the L2 change in the generated image. It then correlates per-image and per-pixel scores (aggregated attention vs. conditional mutual information) with the magnitude of the image change. At the image level, CMI achieves a Pearson correlation of 0.34 with the L2 change, compared to 0.24 for attention. At the pixel level the gap is smaller (0.27 vs 0.21). The authors note that a word can receive high attention yet have negligible effect on the output when the context already encodes the same information, demonstrating that attention is not a faithful indicator of causal influence in the model.