Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inference

Tue, 22 Sep 2026 00:00:00 +0900

Paper

How Many Prefills and How Many Decodes Within a Power Budget: The Capacity-Power Pareto Front for PD-Disaggregated Inference

One-Line Summary (TL;DR)

This paper derives closed-form expressions for how provisioning $(n_P, n_D)$ in prefill-decode disaggregated inference determines serving capacity $\mu$ (req/s) and average power $P$ (W), and shows how to select the minimum-power configuration on the capacity-power Pareto front over candidate deployments (source: §2.2, §2.3). The key is modeling the coupling in which decode-side KV cache is shared between active decode requests and reservations for requests waiting for and running prefill, as a memory balance (source: §3.1.3, Fig. 2). Serving Qwen3-32B (32.8B params) on H200 GPUs (peak 989 TFLOP/s, 4.8 TB/s) with SGLang 0.5.9, the experiments achieve capacity errors of 1.2% (fixed length, 4096 input tokens / 256 output tokens), 3.0% (Mooncake trace), and 1.6% (Azure trace), and power errors of 2.3%, 2.7%, and 2.6% (source: §5.2, Tab. 2, Tab. 3).

Performance on Jaehun's Blog

Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inference

How Many Prefills and How Many Decodes Within a Power Budget: The Capacity-Power Pareto Front for PD-Disaggregated Inference

One-Line Summary (TL;DR)