Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data
Introduction In settings where the batch size is close to 1, such as conversational services, the latency of a mixture-of-experts language model comes mostly from reading…
Mixture of Experts
13 pages
Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation
Problem: Why It's Hard to Unify Understanding and Generation in One Representation There have been many attempts to merge image understanding and image generation into a…
TokenRouter: Efficient Serving System for Token-Level LLM Routing
TokenRouter: An Efficient Serving System for Token-Level LLM Routing TL;DR — Routing an LLM at the token level rather than the session/query level can substantially push…
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
TRACE: Rollout-Aligned FP4 Quantization-Aware Training — MoE LLM RL at BF16 Quality, up to 5.4x Faster TL;DR In reinforcement learning (RL) for MoE LLMs, quantizing…
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
DeepSeek-V4.1-Flash: A Multimodal MoE That Cuts the KV Cache to 1/4 While Raising Performance Further TL;DR. DeepSeek-V4.1-Flash, released by DeepSeek-AI, is a multimodal…
The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
Waking the MoE on SSD: Serving a 35B MoE at 20 tok/s on a 24 GB Desktop with Edge0 One-line summary (TL;DR) Even at 4-bit, a 35B-class MoE occupies 19.5 GB and cannot…
Miles v0.1: Production-Level Post-Training
Miles v0.1: A "verified, clean, and scalable" full-stack system for frontier RL post-training TL;DR: Miles is a full-stack RL post-training framework that resolves the…
Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference
Cache-Learning Routers: Cache-Aware Joint Router Adaptation for Memory-Constrained MoE Inference TL;DR Starting from the observation that MoE's real bottleneck is not…
SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
SMELT: Scaling Laws for Compute-Matched MoE Loop Transformers TL;DR Loop transformers scale depth by executing a layer block repeatedly, but most prior work fixed only…
![[Paper Review] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models](https://paper-assets.alphaxiv.org/figures/2601.07372v1/img-0.jpeg)
[Paper Review] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
Engram: The Second Sparsity Axis After Conditional Computation (MoE), Conditional Memory One-Line Summary (TL;DR) Engram adds a "knowledge lookup primitive" to the…
![[Paper Review] NVIDIA Nemotron 3: Efficient and Open Intelligence](https://developer-blogs.nvidia.com/wp-content/uploads/2025/12/image3-8-png.webp)
[Paper Review] NVIDIA Nemotron 3: Efficient and Open Intelligence
NVIDIA Nemotron 3: Pushing the "Accuracy/Throughput" Frontier with a Hybrid Mamba–Transformer MoE Nemotron 3 combines an MoE hybrid Mamba–Transformer, LatentMoE, MTP,…
![[Paper Review] Qwen 3 Technical Report](https://qianwen-res.oss-accelerate-overseas.aliyuncs.com/logo_qwen3.png){kind=logo}
[Paper Review] Qwen 3 Technical Report
Qwen 3: The Evolution of a Giant MoE Language Model with Adjustable Reasoning Depth TL;DR (in one line) Qwen 3 couples a user-controllable Thinking Budget with a…
![[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding](https://www.storagereview.com/wp-content/uploads/2025/07/image2-2-png-e1752234784623.webp)
[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding
Helix Parallelism: Breaking the Latency-Throughput Wall of Ultra-Long LLM Decoding TL;DR Helix Parallelism schedules Attention and FFN with different parallelism…