Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

Mon, 07 Sep 2026 00:00:00 +0900

Paper

The Recurrent Half Is the Easy-to-Quantize Half: Why Gated DeltaNet Survives at 4 Bits

TL;DR — The 48 recurrent Gated DeltaNet (GDN) layers of a hybrid 27B LLM (Qwen3.8-27B) have until now been protected at 8–16 bits, on the intuition that “error accumulates across the context.” This paper shows that intuition is exactly backwards. Minima, which quantizes all 496 layers — GDN gates included — to NVFP4 W4A4, matches BF16 within seed noise on 6 benchmarks (5-task average −0.52) while also being the smallest (17.5 GiB) and the fastest at prefill (+14–19%). On top of that, a four-stage mechanistic study delivers an architecture-level account of why — block scaling localizes outliers, gate nonlinearity compresses noise, and the delta rule actively erases state error.

2609.04098 on Jaehun's Blog

Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

The Recurrent Half Is the Easy-to-Quantize Half: Why Gated DeltaNet Survives at 4 Bits