Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

What Happened

The paper presents a method that reformulates transformer block removal as a constrained binary optimization (CBO) problem mapped onto an Ising glass, using a Hessian matrix derived from a second‑order Taylor expansion to capture pairwise block couplings. The approach allows fast energy calculations for candidate pruning configurations, enabling brute‑force search for small models and quantum‑inspired solvers for larger ones. Experiments on Llama‑3.3‑70B‑Instruct show that at 50% depth pruning (40/80 blocks removed) the CBO method retains ~77 MMLU, outperforming the block‑influence baseline which drops to the mid‑50s. The method also generalizes to heterogeneous architectures like NVIDIA‑Nemotron‑3‑Nano‑30B‑A3B‑FP8, where it finds better pruning configurations than baseline techniques.

Why It Matters

Enterprise architects can achieve significant inference speedups and memory savings by pruning entire transformer blocks without retraining, reducing operational costs. The Ising‑based optimization provides a systematic way to explore many‑body interactions, potentially improving model reliability and performance in deep‑compression scenarios. Integrating this technique into existing compression pipelines (quantization, low‑rank, width pruning) can streamline model deployment and governance.

The Limitation

The method relies on a pre‑computed Hessian from a small calibration dataset; its accuracy may degrade if the dataset is not representative of production workloads.

Source