Transformer Optimization

3 pages

SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference

SparseDecoding calibrates pruning on model-generated tokens and adds a decoding-specific sparse kernel. We examine the conditions behind its maximum 1.48× speedup on A100…

October 12, 2026

16 min

Model Compression Efficient Inference

[paper review] SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-bit Training

Separate SageAttention3 FP4 inference from SageBwd INT8 training, with checked examples of quantization error, storage overhead and kernel versus whole-workload speed.

July 9, 2025

2 min

Transformer Optimization Inference Acceleration

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Native Sparse Attention (NSA) — 11× faster even at 64k tokens, accuracy intact One-line summary (TL;DR) NSA combines a three-branch sparse attention — “compress → select…

July 7, 2025

36 min

Long Context Transformer Optimization