Tailoring the Quantization Space for 1-Bit KV Cache Compression

Tue, 06 Oct 2026 00:00:00 +0900

Paper

TaSQ: ‘Tailored Design’ of the Quantization Space for a 1-bit KV Cache

TL;DR — TaSQ (Tailored Space Vector Quantization) redesigns the very target space to which vector quantization (VQ) is applied, compressing the KV cache to about 1.25 bits per channel while maintaining quality close to BF16. The key is applying query-guided channel weighting, head-shared normalization, and covariance-aware channel grouping to the pre-RoPE key. On a single RTX 6000 Ada, it expands the KV cache pool by 12.48× and raises the maximum batch from 6→84 (14×) and peak throughput by 1.87× (source: §Abstract, Fig. 4).

Vector Quantization on Jaehun's Blog

Tailoring the Quantization Space for 1-Bit KV Cache Compression

TaSQ: ‘Tailored Design’ of the Quantization Space for a 1-bit KV Cache