IC-028SPADE, an abstaining classifier built on top of ResNet, ViT, and VGG models, detects out-of-distribution and adversarial samples with provable guarantees.
Nicolas Atienza, Johanne Cohen, Christophe Labreuche, Michele Sebag
The paper introduces SPADE, a method that fits a Generalized Extreme Value (GEV) model to the latent distances of a pre-trained teacher model's training distribution. This GEV model is used to define an abstaining classifier that rejects samples deemed extreme. The method is empirically validated on ResNet, VGG, and ViT architectures using CIFAR-10, CIFAR-100, and ImageNet-1K datasets. Results show that SPADE performs comparably or better than state-of-the-art methods like MSP, ODIN, MDS, kNN, and CIDER for both OOD and adversarial sample detection.
Evidence
correlational
Key metric
For OOD detection on ImageNet-1K with ResNet, SPADE achieves AUROC/FPR95 scores of 61.91/85.27 (SSB-hard), 77.99/61.04 (NINCO), 85.26/44.84 (iNaturalist), 95.86/24.63 (Textures), 85.79/46.33 (OpenImages-O), ranking second overall. For adversarial detection (FGSM, ε=0.001-0.004) on CIFAR-10, SPADE achieves average AUROC/FPR95 of 86.10/54.43, ranking first in AUROC.
Caveat
The theoretical guarantees for adversarial robustness depend on the Lipschitz constant of the teacher model, which can lead to trivial bounds. The authors mitigate this with a local Lipschitz constant analysis.