Magnitude Pruning

Findings

• IC-747SOTA pruning methods (SparseGPT, Wanda, magnitude) cause significant degradation on knowledge-intensive tasks for Vicuna and Llama models at 25-30%+ unstructured sparsity, and fail completely for n:m structured sparsity [primary]
• IC-748Pruned LLMs at ≥50% sparsity remain robust in-context retrievers and summarizers, with Vicuna-7B matching up to ~40% sparsity and Vicuna-13B up to ~50% sparsity in open-book settings [primary]
• IC-749Compressed Vicuna-13B at 46.16% sparsity (matching 7B parameter count) achieves lower MMLU accuracy than dense Vicuna-7B, indicating large-sparse models do not outperform small-dense at matched size [primary]