https://jaehun.me/en/tags/sequence-compression/Recent content in Sequence Compression on Jaehun's BlogHugoen-USSun, 25 Oct 2026 08:25:35 +0900https://jaehun.me/en/posts/paper-2610-31574v1/Sun, 25 Oct 2026 00:00:00 +0900https://jaehun.me/en/posts/paper-2610-31574v1/<p><a href="https://arxiv.org/abs/2610.31574v1"target="_blank" class="inline-flex items-center gap-1" >Paper<svg class="h-3 w-3 flex-shrink-0" id="external-link" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 3h6v6m-11 5L21 3m-3 10v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"/></svg> </a></p> <h2 id="introduction">Introduction<a href="#introduction" class="heading-anchor" aria-label="Link to this section"></a></h2><p>In models that handle long contexts, the compression operation that reduces multiple tokens to a single vector becomes a performance bottleneck. This paper reframes compression not as a choice of weighted averaging, but as the problem of finding the coefficients of a regression problem defined by the tokens. TTR-Compressor constructs a label-free ridge regression problem from the input tokens and uses its closed-form solution as the compressed representation.</p>