https://jaehun.me/en/categories/attention-kernels/Recent content in Attention-Kernels on Jaehun's BlogHugoen-USSun, 20 Sep 2026 09:55:15 +0900https://jaehun.me/en/posts/paper-2609-15810/Sun, 20 Sep 2026 00:00:00 +0900https://jaehun.me/en/posts/paper-2609-15810/<p><a href="https://arxiv.org/abs/2609.15810"target="_blank" class="inline-flex items-center gap-1" >Paper<svg class="h-3 w-3 flex-shrink-0" id="external-link" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 3h6v6m-11 5L21 3m-3 10v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"/></svg> </a></p> <h2 id="vc-attention-value-smoothing-and-softmax-casting-for-the-last-two-bottlenecks-in-low-bit-attention">VC-Attention: Value Smoothing and Softmax Casting for the Last Two Bottlenecks in Low-Bit Attention<a href="#vc-attention-value-smoothing-and-softmax-casting-for-the-last-two-bottlenecks-in-low-bit-attention" class="heading-anchor" aria-label="Link to this section"></a></h2><p>One-line summary (TL;DR): A training-free kernel that removes both value quantization error and FP32 softmax in low-bit attention for video DiTs, achieving 1.59× attention and 1.19× clip-generation speedups on B200 (source: §4.3, Fig. 6).</p>