![[Paper Review] Hardware-Efficient Attention for Fast Decoding](https://moonlight-paper-snapshot.s3.ap-northeast-2.amazonaws.com/arxiv/hardware-efficient-attention-for-fast-decoding-1.png)
[Paper Review] Hardware-Efficient Attention for Fast Decoding
GTA & GLA: Hardware-Efficient Attention That Breaks the 'Memory-Dominated' Decode TL;DR GTA (key–value tying) and GLA (latent head sharding) raise FLOPs/byte and shrink…
Long-Context Decoding
1 page