The paper analyzes the local geometry of Stable Diffusion's learned manifold using three descriptors: local scaling (ψ), local rank (ν), and local complexity (δ). These descriptors are computed from the input-output Jacobian of the model. The study finds that higher local scaling correlates with increased visual complexity and diversity in generated images, while lower values correlate with more modal, subject-focused images. Local rank is associated with the dimensionality of the manifold and background texture. Memorized prompts show distinct trajectories in these descriptors compared to random prompts, and classifier-free guidance scales affect these geometric properties. The analysis covers 50k ImageNet images and COCO prompts.
Evidence
correlational
Key metric
Local scaling (ψ), local rank (ν), local complexity (δ); Vendi score diversity and RAHF human preference scores reported for bins of local scaling; statistical significance (t-test) of differences between in-domain and out-of-domain descriptor distributions; memorized prompts show distinct mean geometry compared to COCO prompts (confidence intervals shown in plots, no specific numbers quoted in text for memorization)
Caveat
The method relies on approximations for large models (randomized SVD, first-order Taylor for non-CPWL activations) and computational complexity is a limitation.