Key Takeaways
- Casting block removal as a constrained Ising spin-glass problem hits 76.9 MMLU with 40 of 80 blocks deleted, a 23-point jump over prior pruning baselines and no retraining.
- One Hessian coupling matrix is computed once and reused across compression ratios, and low-lying excited states can beat the ground state after light retraining.
- Multiverse Computing’s $570M Series C and open CompactifAI code recast compression as thermodynamic search rather than independent per-layer importance scoring.
Table of Contents
A 23-Point MMLU Gap Hides Inside a Spin Glass
On September 21, Hugging Face’s research blog published a method from Multiverse Computing that treats transformer block removal as an energy-minimization problem.
At 40 of 80 blocks removed, the optimization-based method scores 76.9 on MMLU, while the strongest prior block-removal baseline scores 54.0.
That is a 23-point gap without retraining, and it comes from replacing block-by-block importance scores with a full set of pairwise couplings between layers.
Turning Block Deletion Into an Energy Minimization Task
Conventional depth pruning methods, as explained in the Hugging Face research blog, score each transformer block independently, then cut the least promising ones.
Multiverse Computing’s researchers describe that as a mean-field shortcut: it assumes the harm from removing block 20 does not change when block 19 or block 24 is also removed.
Delete enough layers at once and those assumptions break.
The new method attaches a binary variable to each transformer block, where 0 means keep and 1 means remove.
A second-order Taylor expansion of the model loss with respect to those variables produces an approximate Hessian matrix.
Diagonal terms measure individual block importance. Off-diagonal entries are the interactions, or couplings, that mean-field methods discard.
Block selection then becomes a constrained binary optimization: choose exactly M of N blocks to remove while minimizing a quadratic energy function.
Physically, the problem is equivalent to finding low-energy states of an Ising glass with a fixed total spin.
Low-energy spin configurations correspond to high-performing pruned models, making energy minimization and benchmark optimization the same search.
The Hessian Is the Real Coupling Map
Computing the Hessian requires forward and backward passes on a small calibration dataset, and it is done only once.
After that, evaluating any candidate pruning configuration requires a single cheap energy calculation, with no need to load the model or run benchmark tasks.
Because the same coupling matrix does not depend on the target compression ratio, one Hessian can be reused for many different values of M.
Exact Search, Then Quantum-Inspired Solvers
For tractable cases, brute force on a single GPU checks up to tens of billions of spin configurations.
The hardest exact case reported involved removing 8 of 80 blocks in Llama-3.3-70B, a space of roughly 29 billion configurations that took about two days.
When enumeration becomes impossible, the constrained binary optimization can be rewritten as a QUBO with the constraint folded into a penalty term.
That same task can then run on quantum annealing, QAOA, tabu search, or specialized branch-and-bound machinery.
The authors report that an open-source tabu solver reliably reaches the lowest-energy states in seconds even on cases they can verify against brute force.
Notably, the goal is not the true ground state. It is a fast route to several good low-energy states, which lowers the bar for lightweight solvers.
Why the 17th Excited State Can Beat the Ground State
Energy is a strong proxy for benchmark quality, but it is not perfect.
That imperfection becomes useful: the method can read off low-lying excited states almost for free, producing a spectrum of candidate prunings instead of a single fragile answer.
A concrete result from Llama-3.1-8B-Instruct at 16 of 32 blocks removed shows the difference.
Most of the top states cut layers near the end of the model, as earlier work would predict.
The 17th excited state, however, is the first to propose removing a block near the beginning.
After light retraining, that configuration outperforms the ground state across several benchmarks.
The team says this directly disproves the common assumption that the best pruning is one consecutive chunk of middle or late layers.
Hybrid Architectures Fit the Same Spin Model
The method does not assume a homogeneous stack of dense transformer blocks.
Researchers tested it on NVIDIA-Nemotron-3-Nano-30B-A3B-FP8, which interleaves Mamba2, attention, and mixture-of-experts layers in a non-uniform pattern.
Removing two or three MoE layers or two attention layers produced configurations that beat block influence on AIME25 and GPQA.
Those results suggest redundancy in hybrid models is real but unevenly distributed, and the coupled search is what finds the disposable layers.
A $570 Million Context for Quantum-Inspired Compression
The technical result lands at a telling moment for Multiverse Computing.
In a September interview on The New Quantum Era, chief scientific officer Román Orús described the company’s larger compression thesis: using tensor-network and quantum-inspired methods to shrink models by 90 to 95 percent without meaningful accuracy loss.
That ambition is now backed by a $570 million Series C at a $1.7 billion pre-money valuation, announced in July and described as Europe’s largest quantum software company.
The commercial pull is not limited to cloud inference.
Orús points to drones, satellites, vehicles, and industrial devices that cannot depend on cloud connectivity as the main reason smaller models are becoming strategic infrastructure.
His framing is unsparing.
‘We are using atomic bombs to kill a mosquito.’
That comment targets overparameterized LLMs, but it also captures why many-body coupling matters: redundancies are not evenly distributed, and most layers are not equally disposable.
Recent open-model releases reinforce the pattern.
HyperNova 60B was created by compressing GPT-OSS-120B by half, and Pulsar 16B launched with NVIDIA in June 2026 with an 87.22 score on AIME 2025.
A May 2026 experiment pushed the technique further: individual layers of Llama 3.1 8B were encoded as quantum circuits and executed on IBM’s 156-qubit Quantum System Two while the model generated text.
The measured 1.4 percent perplexity improvement sounds modest, but Orús argues the correct benchmark is improvement per parameter added, not the headline percentage alone.
In that assessment, the quantum-circuit layers outperformed equivalent classical parameters.
That vendor-specific framing points to a directional shift rather than an immediately deployable production claim.
Block removal also slots alongside quantization, low-rank and SVD compression, width pruning, and knowledge-distillation-based healing, rather than competing with them.
The underlying code is open-sourced in a CompactifAI repository, which lowers the barrier for teams that want to test coupled block removal on their own checkpoints.
Compression Is Now a Thermodynamic Discipline
The deepest compression wins are no longer coming from better importance scores. They come from treating the model as a coupled many-body system and harvesting multiple low-energy solutions instead of chasing a single optimum.
For AI teams refining compressed open-model deployments, programmatic SEO and AI automation is how Andres SEO Expert approaches scaling technical visibility — contact the studio.
Frequently Asked Questions
What is the key innovation in Multiverse Computing’s block-removal method?
It treats transformer block removal as an energy-minimization problem on an Ising spin glass with pairwise couplings between layers. Instead of scoring blocks independently, a second-order Taylor expansion of the loss produces a Hessian whose off-diagonal terms capture layer interactions, then a constrained binary optimization selects exactly M of N blocks to remove.
How does the method achieve a 23-point MMLU gap over prior block-removal baselines?
At 40 of 80 blocks removed, the optimization-based method scores 76.9 on MMLU, while the strongest prior block-removal baseline scores 54.0. The gain comes from modeling pairwise couplings between layers rather than relying on independent block importance scores, and it is achieved without retraining.
Why can the 17th excited state beat the ground state for pruning?
Energy is a strong proxy for benchmark quality but not a perfect one. For Llama-3.1-8B-Instruct at 16 of 32 blocks removed, the 17th excited state is the first to remove a block near the beginning of the model, and after light retraining it outperforms the ground state across several benchmarks. This shows the best pruning is not always one consecutive middle or late chunk.
What solvers are used when exact search becomes impossible?
For tractable cases, brute force on a single GPU checks up to tens of billions of spin configurations. For harder cases, the constrained binary optimization is rewritten as a QUBO with the constraint folded into a penalty term, allowing quantum annealing, QAOA, tabu search, or branch-and-bound. An open-source tabu solver reliably reaches the lowest-energy states in seconds.
Does the method work on hybrid architectures such as Mamba2 and mixture-of-experts?
Yes. Researchers tested it on NVIDIA-Nemotron-3-Nano-30B-A3B-FP8, which interleaves Mamba2, attention, and mixture-of-experts layers in a non-uniform pattern. Removing two or three MoE layers or two attention layers produced configurations that beat block influence on AIME25 and GPQA, suggesting hybrid-model redundancy is real but unevenly distributed.
What is the commercial context for this quantum-inspired compression work?
Multiverse Computing has raised a $570 million Series C at a $1.7 billion pre-money valuation. Chief scientific officer Román Orús describes a larger thesis of using tensor-network and quantum-inspired methods to shrink models by 90 to 95 percent without meaningful accuracy loss, targeting drones, satellites, vehicles, and industrial devices that cannot depend on cloud connectivity. The underlying code is open-sourced in a CompactifAI repository.
How does block removal relate to quantization, low-rank compression, and distillation?
Block removal slots alongside quantization, low-rank and SVD compression, width pruning, and knowledge-distillation-based healing rather than competing with them. The broader shift is to treat compression as a thermodynamic discipline: model the network as a coupled many-body system, harvest multiple low-energy solutions, and combine block removal with other compression and recovery techniques.
