One Shot vs. Iterative: Rethinking Pruning Strategies for Model Compression

The Pruning Pendulum: Why One-Shot Compression Might Outweigh Iterative Methods

A new paper on arXiv (2508.13836v2) revisits a foundational question in model compression: should you prune a neural network all at once or gradually over multiple training cycles? The researchers systematically compare one-shot pruning—where a model is trained, pruned in a single pass, and then fine-tuned—against iterative pruning, which alternates between pruning small amounts and retraining. The findings challenge a long-standing assumption that iterative methods are inherently superior.

What the Research Reveals

The study provides rigorous empirical evidence that one-shot pruning can match or even exceed the performance of iterative approaches under many practical conditions. This matters because the AI industry has largely defaulted to iterative pruning as the safer, more reliable option. The paper demonstrates that with proper initialization and post-pruning fine-tuning, a single pruning step achieves comparable compression ratios without the computational overhead of multiple training cycles.

Crucially, the analysis controls for variables often conflated in prior work: pruning schedule, learning rate adjustments, and the amount of total compute budget. When total training compute is held constant, one-shot methods frequently outperform iterative ones—a counterintuitive result that suggests iterative pruning may waste compute on recovering from repeated structural disruptions.

Why This Shifts the Ground for Practitioners

For teams deploying large language models or vision transformers, this is not an academic nuance. Iterative pruning can multiply training time by a factor of three to five, depending on the number of pruning rounds. If one-shot methods are viable, organizations can achieve the same compressed model with significantly lower energy costs and faster iteration cycles.

The finding also has implications for deployment pipelines. Iterative pruning requires careful checkpoint management and scheduling logic; one-shot pruning simplifies the workflow to a single training run followed by a single pruning and fine-tuning step. This reduces engineering complexity and the risk of training instability from repeated weight rewinding.

However, the paper does not claim one-shot is universally superior. The effectiveness depends on the pruning ratio—at very high sparsity levels (above 90%), iterative methods still show an edge. The key insight is that for the compression ratios most practitioners target (50-80% sparsity), one-shot is often the better choice.

Implications for AI Practitioners

This research should prompt teams to re-evaluate their default compression strategies. Many organizations have baked iterative pruning into their tooling based on older literature. The burden of proof should now shift: unless you are targeting extreme sparsity, start with a well-tuned one-shot approach before committing to iterative cycles.

The paper also underscores the importance of fine-tuning quality over pruning schedule. A single, well-executed fine-tuning phase after pruning can compensate for the lack of gradual weight adjustments during the pruning process itself.

Key Takeaways

• One-shot pruning matches or outperforms iterative pruning at moderate sparsity levels (50-80%), challenging conventional wisdom in model compression.
• Iterative pruning’s advantage only clearly emerges at extreme sparsity ratios above 90%, making it a niche rather than default choice.
• One-shot methods reduce training compute by 3-5x and simplify deployment pipelines, offering immediate cost and engineering benefits.
• Practitioners should benchmark one-shot pruning first for their target compression ratios before investing in iterative schedules.