Cache Engineering for Agents
The Cost and Accuracy Dilemma of Long Agent Trajectories LLM agents accumulate long trajectories by repeating reasoning and tool calls for hundreds of rounds. To make…
Kv-Cache
19 pages
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Fixed-point behavior connects KV sharing, state reuse, and distilled prefill in looped language models. We review the savings and accuracy trade-offs demonstrated on…
SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention
SlimKV: Compressing Tokens and Features Together, Cutting the KV Cache by up to 32× with Reconstruction-Free Latent-Space Attention TL;DR SlimKV is a question-agnostic…
Tailoring the Quantization Space for 1-Bit KV Cache Compression
TaSQ: 'Tailored Design' of the Quantization Space for a 1-bit KV Cache TL;DR — TaSQ (Tailored Space Vector Quantization) redesigns the very target space to which vector…
Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
How to Hand Off KV Caches As-Is Between Different LLMs: HeteroFold TL;DR: HeteroFold, a method that reuses the KV cache between models with different tokenizers and…
The KV Cache Is the New Memory Wall
The KV Cache Is the New Memory Wall: Three Regimes and Five Domains of Long-Context LLM Inference TL;DR — Long-context LLM decode inference is bound by memory bandwidth,…
Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
Phase Sensitivity: The 'Periodic Weakness' Created by Chunked KV-Cache Compression TL;DR — In models that use chunked KV-cache compression (such as the DeepSeek-V4…
WorldAttention: An Efficient Attention Architecture for Interactive Video World Models
WorldAttention: System Co-Design of an Interactive Video World Model That Drops the Sliding Window TL;DR — Text-controlled interactive video world models are trapped…
Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Flash-dLLM: I/O-First Diffusion LLM Acceleration, Unlocking 210.6 tokens/s with Fused Cache and Self-Verification One-Line Summary (TL;DR) Diffusion LLMs (dLLMs) support…
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
DeepSeek-V4.1-Flash: A Multimodal MoE That Cuts the KV Cache to 1/4 While Raising Performance Further TL;DR. DeepSeek-V4.1-Flash, released by DeepSeek-AI, is a multimodal…
Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches
Fathom: Accelerating 1M-Token Decoding by 1.67x with Query-Adaptive Depth Scanning TL;DR When agent sessions stretch to 1M tokens and dozens of sessions reside…
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
SAS: A Simple Attention Sparsification That Learns Context Ranking Directly Without Distillation TL;DR — In post-training attention sparsification, inject the continuous…
VestigeKV: The NoPE-MLA KV Cache Carries Its Own Eviction Signal in a Vestigial Branch
VestigeKV: The Degenerate RoPE Branch Carries Its Own Cache Eviction Signal TL;DR In NoPE-MLA models (Kimi Linear 48B), reading only 11% of each row — the 64-dim…
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
ShallowStream: Index Shallowly, Answer Deeply — Unlocking the Computational Bottleneck of Streaming Video Understanding TL;DR — In streaming video understanding, the most…
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
Catch the Moment a Model "Revisits Its Thoughts": How BeaconKV Compresses Reasoning KV Caches 5.8× with Beacon Queries TL;DR — The KV cache of a Large Reasoning Model…
Language Models Can Control Their Own Attention
Language Models Control Their Own Attention: Declarative Attention TL;DR — Existing sparse attention methods still paid $O(N)$ per decoding step by approximating…
Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving
Same Request, Different Answer: Prefix Caching Makes Serving Non-Reproducible, and Quantization Amplifies It TL;DR — Prefix caching is billed as a "transparent…
SGD-KV: Summarization Guided KV Cache Compression
SGD-KV: Finding 'heads that are good at summarizing' cuts the 1M-token KV cache by up to 75% TL;DR — Attention heads do not all do the same job. SGD-KV pinpoints the…
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
The Scores Were a Waste: The Paradox Where 'Random Sampling' Ties the Strongest in KV Cache Eviction — An In-Depth Review of Random Attention TL;DR — Research on…