TL;DR — Research on reasoning-specific KV cache eviction has always been the problem of scoring “which tokens will matter later” and picking the top-K. Random Attention forcibly preserves only the prompt with +∞ and evicts the rest without any scores, uniformly at random per head. Across 4 models and 6 reasoning tasks it matches the strongest baseline (TriAttention) in accuracy (statistical advantage in 31 of 60 comparison cells, significantly worse in just 1), and since vLLM serving has no scoring pass it delivers 32–43% higher throughput (source: §1, Fig. 1, Tab. 1, Tab. 4).

With-Qwen3-8-Flash on Jaehun's Blog

Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning

The Scores Were a Waste: The Paradox Where ‘Random Sampling’ Ties the Strongest in KV Cache Eviction — An In-Depth Review of Random Attention