Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
Paper How to Hand Off KV Caches As-Is Between Different LLMs: HeteroFoldTL;DR: HeteroFold, a method that reuses the KV cache between models …
20 min
KV Cache
Efficient Inference
Long Context
Large Language Models
Multiagent Systems
LLM Inference