https://jaehun.me/en/tags/prefill/Technical notes by Jaehun Ryu, a deep learning compiler engineer, on LLM inference, KV caches, compilers, C++, AI papers, and open-source code.en-UShttps://jaehun.me/en/posts/paper-2611-09709v1/https://jaehun.me/en/posts/paper-2611-09709v1/Thu, 12 Nov 2026 00:00:00 +0900<p><a href="https://arxiv.org/abs/2611.09709v1"target="_blank" class="inline-flex items-center gap-1" >Paper<svg class="h-3 w-3 flex-shrink-0" id="external-link" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 3h6v6m-11 5L21 3m-3 10v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"/></svg> </a></p> <h2 id="introduction-the-wait-before-an-answer-appears">Introduction: The wait before an answer appears<a href="#introduction-the-wait-before-an-answer-appears" class="heading-anchor" aria-label="Link to this section"></a></h2><p>With a long prompt, the model must pass the entire document through before the first token emerges, and the user waits the whole time seeing nothing. To cut that wait, RidgeFill skips computation in some layers and prefills the cache those layers would have produced by approximating it. No training is needed, and the tail, some selected rows, and every decoding step are computed exactly.</p>