GDN Tree-Scan: Served Tree Verification for Recurrent-Hybrid Language Models
LumoTree: Path-Parallel Speculative Verification for Hybrid Language Models TL;DR — Tree speculative decoding in hybrid models that mix Gated DeltaNet with full attention…
SSM
4 pages
Kalman Delta Networks: Uncertainty-aware Associative Memory
Linear Attention Rewritten with a Kalman Filter: Kalman Delta Networks TL;DR The fixed-size recurrent memory update of linear attention is reinterpreted as a…
![[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs](https://pbs.twimg.com/media/GdyLXO9W4AADox0.jpg)
[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs
Marconi: Rethinking Prefix Caching for the Hybrid LLM Era TL;DR Marconi introduces a prefix-caching framework for hybrid LLM inference (Attention + SSM). It selectively…
![[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://icml.cc/media/PosterPDFs/ICML%202024/32613.png)
[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
Structured State Space Duality: Unifying SSMs and Attention with Mamba-2 for 2–8× Acceleration TL;DR Structured State-Space Duality (SSD) reveals that SSMs and masked…