Key Takeaways
- QAD Q4_0 GGUFs restore 97% of BF16 accuracy across four LFM2.5 checkpoints.
- They keep the same Q4_0 memory footprint and deliver 3-33% faster decode on edge hardware.
- Quantization-aware distillation is now a proven route for edge AI, backed by NVIDIA and independent tests.
Table of Contents
Four LFM2.5 Checkpoints Hit Q4_0 Memory Without the Usual Accuracy Crash
Hugging Face reports that Liquid AI has shipped QAD Q4_0 GGUFs across four LFM2.5 checkpoints, recovering 97% of the BF16 average accuracy lost to quantization.
The release covers LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B, with the same memory footprint and decode throughput as native Q4_0 artifacts.
For edge developers, that removes the hardest tradeoff: low-bit models have historically traded precision for throughput.
How Quantization-Aware Distillation Rebuilds the Q4_0 Tradeoff
Quantization-aware distillation changes the sequence. Instead of compressing a finished model and accepting the loss, a high-precision teacher supervises a quantized student during training.
The student learns to mimic the teacher’s behavior under simulated low-precision constraints, which preserves more of the original representation.
Recovery: 97% of their BF16 average accuracy lost to quantization is recovered
According to the Hugging Face blog, Liquid AI evaluated the QAD checkpoints against post-training quantization GGUFs across GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4.
The BF16 GGUF served as the in-format ceiling, while GSM8K and AIME25 provided scale-appropriate math evaluation for smaller and larger models.
Across the four releases, QAD checkpoints retained 97.1%, 96.5%, 97.4%, and 96.6% of their BF16 baselines.
On real edge hardware, the results sharpen.
The team profiled decode throughput on MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5.
MacBook Pro and NucBox used GPU inference, while Samsung and Raspberry Pi ran Arm CPU inference.
- 230M and 350M: QAD Q4_0 matched Q5_K_M quality within evaluation variance at 4-33% higher decode throughput.
- 1.2B and 2.6B: QAD Q4_0 matched Q4_K_M quality at 3-14% higher throughput.
Where applicable, the 230M and 1.2B checkpoints also matched Unsloth’s UD-Q4_K_XL, a strong external post-training quantization baseline.
Developers can load the files through llama.cpp or any GGUF Q4_0-compatible runtime via Hugging Face.
Why Edge AI Benchmark Leaders Should Watch the QAD Pattern
The same distillation-to-quantization sequence is now showing up beyond Liquid AI’s release.
The NVIDIA Developer Blog documents a two-stage QAD pipeline for Nemotron 3.5 Lightning NVFP4, compressing a 66 GB BF16 checkpoint to 22 GB while reporting up to 4x higher throughput.
That 4x figure is vendor-reported and has not yet been independently validated across all production edge environments.
In NVIDIA’s checkpoint examples, QAD pushed median score recovery from 96.33% with aggressive PTQ to 99.72% on one configuration, using the same 21.19 GB footprint.
Another checkpoint showed a smaller final gap, but QAD still gained on agentic and coding evaluations such as Terminal-Bench v2.1.
Liquid AI’s published benchmark suite points in the same direction: quantization-aware distillation compresses the quality gap without changing the deployment envelope.
Real-world edge throughput remains the harder test.
MDPI Applied Sciences evaluated a related LFM2-24B-A2B Q4_K_M GGUF on an NVIDIA RTX 5060 Ti, recording 39.20 tokens per second and 0.926 semantic similarity on SQuAD.
Its native Pythonic function-calling syntax achieved 90.0% format success, while forced JSON parsing dropped format success to 37.5%.
That gap matters for agentic edge applications where schema enforcement can silently degrade model performance.
For teams shipping local agents or on-device assistants, QAD Q4_0 checkpoints may reduce the need to choose between speed, memory, and instruction-following reliability.
The Edge Deployment Equation Just Shifted
Low-bit edge models no longer have to accept the old precision penalty, and the benchmark data now gives engineering teams a concrete reason to re-evaluate Q4_0 deployments.
For teams tracking edge model compression and AI infrastructure shifts, programmatic SEO AI automation is how Andres SEO Expert helps technical brands stay ahead of these releases — contact the team.
Frequently Asked Questions
What is QAD Q4_0 and why is it significant for edge AI?
QAD Q4_0 refers to quantization-aware distillation applied to Q4_0 GGUF models. It trains a quantized student model under simulated low-precision constraints while a high-precision teacher supervises it, recovering 97% of the BF16 accuracy typically lost during quantization while keeping the same memory and throughput.
Which LFM2.5 checkpoints are available as QAD Q4_0 GGUF files?
The release covers LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B, all with the same memory footprint and decode throughput as native Q4_0 artifacts.
How much accuracy does QAD recover compared to the BF16 baseline?
Across the four releases, QAD checkpoints retained 97.1%, 96.5%, 97.4%, and 96.6% of their BF16 baselines, meaning roughly 97% of the average accuracy lost to standard quantization is recovered.
How does QAD Q4_0 compare to higher-bit formats like Q5_K_M and Q4_K_M?
For 230M and 350M models, QAD Q4_0 matched Q5_K_M quality within evaluation variance while offering 4-33% higher decode throughput. For 1.2B and 2.6B models, it matched Q4_K_M quality with 3-14% higher throughput.
What hardware was used to profile QAD Q4_0 decode throughput?
The team profiled performance on MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5. MacBook Pro and NucBox used GPU inference, while Samsung and Raspberry Pi ran Arm CPU inference.
Is quantization-aware distillation only used by Liquid AI?
No. NVIDIA has documented a two-stage QAD pipeline for Nemotron 3.5 Lightning NVFP4, compressing a 66 GB BF16 checkpoint to 22 GB with reported up to 4x higher throughput and improved median score recovery compared to aggressive post-training quantization.
