FLOP-Aware Scheduling

[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs

Marconi: Rethinking Prefix Caching for the Hybrid LLM Era TL;DR Marconi introduces a prefix-caching framework for hybrid LLM inference (Attention + SSM). It selectively…