![[Paper Review] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models](https://paper-assets.alphaxiv.org/figures/2601.07372v1/img-0.jpeg)
Engram: The Second Sparsity Axis After Conditional Computation (MoE), Conditional Memory One-Line Summary (TL;DR) Engram adds a "knowledge lookup primitive" to the…
![[Paper Review] Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin](https://cdn-uploads.huggingface.co/production/uploads/6317233cc92fd6fee317e030/yNP71PjobVvLDgJ0R0qV2.png)
One Narrative Forged by Massive Activations: Bridging Attention Sink and Compression Valley TL;DR Massive activations in the residual stream (especially BOS) form a…
![[Paper Review] Hardware-Efficient Attention for Fast Decoding](https://moonlight-paper-snapshot.s3.ap-northeast-2.amazonaws.com/arxiv/hardware-efficient-attention-for-fast-decoding-1.png)
GTA & GLA: Hardware-Efficient Attention That Breaks the 'Memory-Dominated' Decode TL;DR GTA (key–value tying) and GLA (latent head sharding) raise FLOPs/byte and shrink…
"Let's put an OS on the GPU": A proposal for a GPU multitasking OS layer for the LLM era One-line summary (TL;DR) In LLM serving, GPUs often stay at ≤10% utilization…
![[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs](https://pbs.twimg.com/media/GdyLXO9W4AADox0.jpg)
Marconi: Rethinking Prefix Caching for the Hybrid LLM Era TL;DR Marconi introduces a prefix-caching framework for hybrid LLM inference (Attention + SSM). It selectively…
![[Paper Review] SGLang: Efficient Execution of Structured Language Model Programs](https://cdn.bytez.com/mobilePapers/v2/neurips/94872/images/20-0.png)
SGLang & RadixAttention: How Execution Optimization for "LM Programs" Achieved a 6.4x Speedup TL;DR By combining a language-runtime co-design with RadixAttention (radix…
![[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://icml.cc/media/PosterPDFs/ICML%202024/32613.png)
Structured State Space Duality: Unifying SSMs and Attention with Mamba-2 for 2–8× Acceleration TL;DR Structured State-Space Duality (SSD) reveals that SSMs and masked…
![[Paper Review] Inference-Time Hyper-Scaling with KV Cache Compression](https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05345/gradient.png)
Dynamic Memory Sparsification (DMS): Making LLM Hyper-Scaling a Reality with 8× KV Cache Compression One-Line Summary (TL;DR) DMS, combining a lightweight re-fitting of…
![[Paper Review] Llama-Nemotron: Efficient Reasoning Models](https://cdn-thumbnails.huggingface.co/social-thumbnails/collections/nvidia/llama-nemotron-67d92346030a2691293f200b.png)
Hydragen: The Secret Weapon for Decoding Large Batches with Shared Prefixes up to 32× Faster TL;DR By decomposing the prefix and suffix using softmax denominator…
![[Paper Review] KIMI K2: OPEN AGENTIC INTELLIGENCE](https://github.com/MoonshotAI/Kimi-K2/raw/main/figures/kimi-logo.png){kind=logo}
Kimi K2: An Open-Source LLM’s Leap Toward Agentic Intelligence TL;DR With a 3-stage pipeline consisting of MuonClip pretraining + large-scale agentic tool-use data +…
![[Paper Review] Qwen 3 Technical Report](https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/logo_qwen3.png){kind=logo}
Qwen 3: The Evolution of a Giant MoE Language Model with Adjustable Reasoning Depth TL;DR (in one line) Qwen 3 couples a user-controllable Thinking Budget with a…
![[Paper Review] Massive Activations in Large Language Models](https://eric-mingjie.github.io/massive-activations/assets/main_teaser_final.png)
Massive Activations, Hidden Biases: A Reinterpretation of Self-Attention’s Secrets TL;DR Just 4–10 extreme scalar values (×10,000) out of tens of millions can…
Peri-LayerNorm: A Third Option Beyond Post-LN and Pre-LN TL;DR By simply adding another LayerNorm right after the residual connection—"Peri-LayerNorm (Peri-LN)"—the…
![[paper review] SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-bit Training](/assets/images/papers/2505.11594/figure-2.png)
Separate SageAttention3 FP4 inference from SageBwd INT8 training, with checked examples of quantization error, storage overhead and kernel versus whole-workload speed.
![[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding](https://www.storagereview.com/wp-content/uploads/2025/07/image2-2-png-e1752234784623.webp)
Helix Parallelism: Breaking the Latency-Throughput Wall of Ultra-Long LLM Decoding TL;DR Helix Parallelism schedules Attention and FFN with different parallelism…
Subgoal Curriculum + CoT Consistency: DeepSeek-Prover-V2 Reshapes Automated Theorem Proving TL;DR DeepSeek-Prover-V2, built on the principle of "breaking a problem into…
Inference-Time Scaling: How DeepSeek-GRM Surpassed Giant Models One-Line Summary (TL;DR) "27B model × 32 samples"—With only a Generative Reward Model (GRM) and k-Vote…
CODE I/O: From Code I/O + Natural-Language CoT to General-Purpose Reasoning — Lifting 7B-30B LLMs by +2 Points on Average with Data Design Alone TL;DR A single data…
Native Sparse Attention (NSA) — 11× faster even at 64k tokens, accuracy intact One-line summary (TL;DR) NSA combines a three-branch sparse attention — “compress → select…
Janus-Pro 7B: Dual-Encoder Multimodal LLM That Outsmarts Bigger Models One-line summary (TL;DR) By fully separating the SigLIP understanding encoder from the VQ…
One-line Summary (TL;DR) DeepSeek-V3 is an open-source SOTA that combines Aux-loss-free Load-Balancing Bias, FP8 mixed-precision training, and Multi-Token Prediction in a…
One-line summary (TL;DR) JanusFlow achieves state-of-the-art performance in both image understanding and generation (FID 9.51 / GenEval 0.63 / MMBench 74.9) with a single…