Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
Native Sparse Attention (NSA) — 11× faster even at 64k tokens, accuracy intact One-line summary (TL;DR) NSA combines a three-branch sparse attention — “compress → select…