SourceWhat Makes a Good Prune? Maximal Unstructured Pruning for Maximal Cosine Similarity
The paper applies its proposed optimal L1 pruning method to three PyTorch pre-trained ImageNet models. While VGG11 and ResNet18 maintain performance after pruning at the Pareto-optimal point, MobileNetV2 suffers dramatic degradation. The mechanism is that MobileNetV2's parameter distribution has a very high kurtosis of layer-wise kurtoses (64.40 versus 1.42 for VGG11 and 5.81 for ResNet18), meaning it contains layers composed almost entirely of low-magnitude parameters. Global L1 pruning removes these small parameters first, eventually zeroing entire layers and disconnecting them from the network. A conservative adjustment (multiplying the optimal pruning percentage by 1/ln κ²) reduces the pruning to 21% for MobileNetV2 versus 51% for ResNet18.