GPU Acceleration

2 pages

[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

Paper Link Structured State Space Duality: Unifying SSMs and Attention with Mamba-2 for 2–8× AccelerationTL;DRStructured State-Space Duality …

October 3, 2025

Mamba SSM Transformer Long Context Efficient Training FlashAttention GPU Acceleration

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Paper Native Sparse Attention (NSA) — 11× faster even at 64k tokens, accuracy intactOne-line summary (TL;DR)NSA combines a three-branch …

Long Context Transformer Optimization GPU Acceleration FlashAttention Memory Efficiency DeepSeek