The paper validates the Manifold Memorization Hypothesis by showing that estimates of the local intrinsic dimension of the model's learned manifold are lower for memorized images than for non-memorized ones. For Stable Diffusion v1.5, FLIPD estimates of LIDθ (both unconditional and conditional) and the CFG vector norm all assign smaller values to 86 memorized LAION images compared to 4251 non-memorized images. For CIFAR-10, the Normal Bundle estimator (IDDPM) and Jacobian rank (StyleGAN2-ADA) show the same pattern: memorized samples have lower LIDθ, and exact memorization corresponds to low LID* in the training data (data-driven memorization), while reconstructive memorization shows low LIDθ despite comparable LID* (overfitting-driven memorization).
Evidence
correlational
Key metric
86 memorized images vs 2000 LAION aesthetics 6.5+ / 2000 COCO / 251 Tuxemon non-memorized images for Stable Diffusion; 50,000 images generated per model on CIFAR-10; CFG analysis at λ = 7.5 and t = 0.02 on 20 memorized and 20 non-memorized images
Caveat
Image complexity confounds LIDθ estimates: simpler images tend to have lower LIDθ regardless of memorization, causing overlap between memorized and non-memorized samples. The authors partially address this with PNG compression normalization, which slightly improves separation.