Tailoring the Quantization Space for 1-Bit KV Cache Compression
Paper TaSQ: ‘Tailored Design’ of the Quantization Space for a 1-bit KV CacheTL;DR — TaSQ (Tailored Space Vector Quantization) …
16 min
KV Cache
Quantization
Efficient Inference
Long Context
Large Language Models
Vector Quantization