https://jaehun.me/en/tags/pipeline-parallelism/Recent content in Pipeline Parallelism on Jaehun's BlogHugoen-USFri, 02 Oct 2026 09:27:51 +0900https://jaehun.me/en/posts/paper-2609-35263v1/Fri, 02 Oct 2026 00:00:00 +0900https://jaehun.me/en/posts/paper-2609-35263v1/<p><a href="https://arxiv.org/abs/2609.35263v1"target="_blank" class="inline-flex items-center gap-1" >Paper<svg class="h-3 w-3 flex-shrink-0" id="external-link" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path fill="none" stroke="currentColor" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M15 3h6v6m-11 5L21 3m-3 10v6a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V8a2 2 0 0 1 2-2h6"/></svg> </a></p> <h2 id="wavepp--a-pipeline-parallel-prefill-runtime-that-burns-in-prefix-reuse">WavePP — A Pipeline-Parallel Prefill Runtime That Burns In Prefix Reuse<a href="#wavepp--a-pipeline-parallel-prefill-runtime-that-burns-in-prefix-reuse" class="heading-anchor" aria-label="Link to this section"></a></h2><h2 id="tldr">TL;DR<a href="#tldr" class="heading-anchor" aria-label="Link to this section"></a></h2><p>Built on top of TensorRT-LLM, <strong>WavePP</strong> overlaps request admission (reuse consensus, cache protection, capacity reservation) <strong>asynchronously</strong> with pipeline execution, and dynamically tunes the wave size to keep the pipeline full. On the <strong>same kernels and the same topology</strong>, it pushed prefill throughput for GLM 5.2 and MiniMax M2.7 up to <strong>2.91×/2.02×</strong>, and across the 28 Kimi K3 settings it recorded the highest throughput in <strong>all 18</strong> with concurrency 8 or higher.</p>