[Paper Review] DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving
Paper Link DroidSpeak: Reducing Prefill Latency by 1.7–3.1× through Cross-LLM Prefix-KV ReuseTL;DRWhen multiple LLMs share the same …
15 min
droidspeak
Cross-LLM KV Reuse
Prefix-KV Cache
contiguous-layer-recompute