WavePP: High-Throughput Pipeline Parallel LLM Prefill under Prefix Reuse
Paper WavePP — A Pipeline-Parallel Prefill Runtime That Burns In Prefix ReuseTL;DRBuilt on top of TensorRT-LLM, WavePP overlaps request …
18 min
KV Cache
Prefix Caching
Efficient Inference
LLM Inference
Pipeline Parallelism
Long Context