Attention Optimization

4 pages

SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

Paper SAS: A Simple Attention Sparsification That Learns Context Ranking Directly Without DistillationTL;DR — In post-training attention …

September 15, 2026

KV Cache Long Context Efficient Inference Attention Optimization Large Language Models Transformer

Kalman Delta Networks: Uncertainty-aware Associative Memory

Linear Attention Rewritten with a Kalman Filter: Kalman Delta NetworksTL;DRThe fixed-size recurrent memory update of linear attention is …

September 12, 2026

KV Cache Long Context Efficient Inference SSM Attention Optimization Kalman Filter

[Paper Review] Hardware-Efficient Attention for Fast Decoding

[Paper Review] Hardware-Efficient Attention for Fast Decoding

Paper GTA & GLA: Hardware-Efficient Attention That Breaks the ‘Memory-Dominated’ DecodeTL;DRGTA (key–value tying) and GLA …

October 23, 2025

Attention Optimization Inference Acceleration KV-Cache Optimization Tensor Parallelism Long-Context Decoding

[Paper Review] Llama-Nemotron: Efficient Reasoning Models

[Paper Review] Llama-Nemotron: Efficient Reasoning Models

Paper Link Hydragen: The Secret Weapon for Decoding Large Batches with Shared Prefixes up to 32× FasterTL;DRBy decomposing the prefix and …

Prefix Caching Efficient Inference Attention Optimization FlashAttention vLLM