IC-990ResNet-50 and DenseNet-101 exhibit a higher mean-to-variance ratio in penultimate pre-ReLU activations for in-distribution samples than for out-of-distribution samples, and the activation-based scaling factor is well-separated between ID and OOD
The paper measures the statistical properties of penultimate-layer activations for pretrained ResNet-50 (ImageNet) and DenseNet-101 (CIFAR-10/100). For both architectures, in-distribution samples consistently show a larger mean/variance ratio than out-of-distribution samples across all tested OOD datasets (SSB-Hard, NINCO, iNaturalist, Textures, OpenImage-O, SVHN, ISUN, Places365). The chi-square test confirms the rectified-Gaussian assumption (all p-values > 0.05). The derived scaling factor r(a) = q(a)/q_p(a) is well-separated between ID and OOD histograms, which is the mechanism by which scaling enhances OOD detection while pruning degrades it.
Evidence
correlational
Key metric
chi-square p-values for Gaussian assumption: ImageNet 0.296, SSB-Hard 0.262, NINCO 0.181, iNaturalist 0.083, Textures 0.099, OpenImage-O 0.155 (all > 0.05); ID μ/σ consistently larger than OOD μ/σ across all OOD datasets (Figs. 2–3)
Caveat
The Gaussian assumption is verified only for the specific OOD datasets tested; the μ/σ difference is shown visually in figures rather than tabulated as exact values. The analysis assumes the final penultimate operation is ReLU.