[Paper Review] DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving

Wed, 08 Oct 2025 00:00:00 +0900

Paper Link

DroidSpeak: Reducing Prefill Latency by 1.7–3.1× through Cross-LLM Prefix-KV Reuse

TL;DR

When multiple LLMs share the same architecture but different weights, DroidSpeak allows the receiving model to reuse the sender model’s prefix KV cache by performing contiguous-layer partial recomputation and E-first pipelining.
This achieves 1.7–3.1× lower TTFT (prefill latency), up to 4× higher online throughput, and negligible quality loss (F1/ROUGE/CodeSim), assuming an offline O(L²) profiling stage (see §4.2–§5.3, Fig.13–15).

Prefix-KV Cache on Jaehun's Blog

[Paper Review] DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving

DroidSpeak: Reducing Prefill Latency by 1.7–3.1× through Cross-LLM Prefix-KV Reuse

TL;DR