Paper Waking the MoE on SSD: Serving a 35B MoE at 20 tok/s on a 24 GB Desktop with Edge0One-line summary (TL;DR)Even at 4-bit, a 35B-class …
Paper StepAudio 3 Gen: Unifying Speech, Song, Music, and Sound Effects with a Single Discrete Autoregressive (RVQ) ModelTL;DRStepAudio 3 …
Paper Is “Lossless” Speculative Decoding Really Lossless? — The Numerical-Precision Trap Exposed by Reproducing OrthrusTL;DR — …
Paper ZGCM-1: How a 7B Dense Model Trades Blows with 235B Frontiers — A Fully Open, Ultra-Efficient Foundation Model Design ReportTL;DR — …
Paper Programmable World Model — Embedding “Executable State” into Generative WorldsOne-Line Summary (TL;DR)Video world models …
Paper SAS: A Simple Attention Sparsification That Learns Context Ranking Directly Without DistillationTL;DR — In post-training attention …
Paper CoVeR: Coverage-Based Pruning That Cuts Visual Tokens of Multi-View 3D Reasoning by up to 92% Using Geometry AloneOne-Line Summary …
Paper Marigold V2: Reviving an Image-Editing Diffusion Transformer (DiT) as a Monocular Depth EstimatorTL;DRThis study fine-tunes an …
Paper SQS: Fusing Pruning and Quantization into One Bayesian Learning — Where Spike-and-Slab Meets GMMTL;DR — Doing pruning and low-bit …
Linear Attention Rewritten with a Kalman Filter: Kalman Delta NetworksTL;DRThe fixed-size recurrent memory update of linear attention is …
Paper X-AuT: Pruning Audio Encoders in Speech LLMs with Behavioral Probes and Cross-Scale DistillationTL;DRX-AuT is a framework that …
Paper Deadline-Filling Prefill Chunks: SLOWeave — An Adaptive Chunking Scheduler for LLM Serving TL;DR — Instead of a fixed prefill chunk …
Paper Why Are Video LLMs Still So Expensive? — A Four-Stage Guide to Inference Efficiency MechanismsTL;DR — This paper classifies 125 …
Paper Miles v0.1: A “verified, clean, and scalable” full-stack system for frontier RL post-trainingTL;DR: Miles is a full-stack …
Paper Two Designs That Make Speculative Decoding Practical for Large-Scale, Long-Context RL Post-TrainingTL;DR : Rollout generation …
Paper Cache-Learning Routers: Cache-Aware Joint Router Adaptation for Memory-Constrained MoE InferenceTL;DRStarting from the observation …
Paper VestigeKV: The Degenerate RoPE Branch Carries Its Own Cache Eviction SignalTL;DRIn NoPE-MLA models (Kimi Linear 48B), reading only 11% …
Paper ShallowStream: Index Shallowly, Answer Deeply — Unlocking the Computational Bottleneck of Streaming Video UnderstandingTL;DR — In …
Paper Uno: AR and Diffusion in One Model — Lossless Parallel Generation for LLM AccelerationTL;DR — LLMs are slow because next-token …
Paper Catch the Moment a Model “Revisits Its Thoughts”: How BeaconKV Compresses Reasoning KV Caches 5.8× with Beacon Queries …
Paper Why Don’t FP4 Tensor Cores Make Attention Faster? Blackwell FlashAttention-4, Solved with Direct-P and Quantized BackpropTL;DR — …
Paper Language Models Control Their Own Attention: Declarative AttentionTL;DR — Existing sparse attention methods still paid $O(N)$ per …
Paper Same Request, Different Answer: Prefix Caching Makes Serving Non-Reproducible, and Quantization Amplifies It TL;DR — Prefix caching is …
Paper SGD-KV: Finding ‘heads that are good at summarizing’ cuts the 1M-token KV cache by up to 75% TL;DR — Attention heads do …
Paper SMELT: Scaling Laws for Compute-Matched MoE Loop TransformersTL;DRLoop transformers scale depth by executing a layer block repeatedly, …
Paper The Recurrent Half Is the Easy-to-Quantize Half: Why Gated DeltaNet Survives at 4 BitsTL;DR — The 48 recurrent Gated DeltaNet (GDN) …
Paper The Scores Were a Waste: The Paradox Where ‘Random Sampling’ Ties the Strongest in KV Cache Eviction — An In-Depth Review …
![[Paper Review] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models](https://paper-assets.alphaxiv.org/figures/2601.07372v1/img-0.jpeg)
Paper Engram: The Second Sparsity Axis After Conditional Computation (MoE), Conditional MemoryOne-Line Summary (TL;DR)Engram adds a …
![[Paper Review] Continuous Autoregressive Language Models](https://discuss.pytorch.kr/uploads/default/original/2X/d/d1107e24375fae20a5f3a4733826880ce5d817db.png)
Paper CALM: Bypassing the Token-by-Token Bottleneck with “Continuous Vector-by-Vector” Likelihood-Free Language ModelingCALM …
![[Paper Review] Memory Retrieval and Consolidation in Large Language Models through Function Tokens](https://moonlight-paper-snapshot.s3.ap-northeast-2.amazonaws.com/arxiv/memory-retrieval-and-consolidation-in-large-language-models-through-function-tokens-4.png)
Paper Function Token Hypothesis: Why Punctuation and Newlines Gate LLM “Memory Retrieval”One-line Summary (TL;DR)This paper …
![[Paper Review] NVIDIA Nemotron 3: Efficient and Open Intelligence](https://developer-blogs.nvidia.com/wp-content/uploads/2025/12/image3-8-png.webp)
Paper NVIDIA Nemotron 3: Pushing the “Accuracy/Throughput” Frontier with a Hybrid Mamba–Transformer MoENemotron 3 combines an …
![[Paper Review] Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation](https://moonlight-paper-snapshot.s3.ap-northeast-2.amazonaws.com/arxiv/radial-attention-onlog-n-sparse-attention-with-energy-decay-for-long-video-generation-2.png)
Paper Radial Attention: O(n log n) Sparse Attention with Energy Decay — Generating “Long Videos” CheaplyOne-Line Summary …
![[Paper Review] Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin](https://cdn-uploads.huggingface.co/production/uploads/6317233cc92fd6fee317e030/yNP71PjobVvLDgJ0R0qV2.png)
Paper One Narrative Forged by Massive Activations: Bridging Attention Sink and Compression ValleyTL;DRMassive activations in the residual …
![[Paper Review] Hardware-Efficient Attention for Fast Decoding](https://moonlight-paper-snapshot.s3.ap-northeast-2.amazonaws.com/arxiv/hardware-efficient-attention-for-fast-decoding-1.png)
Paper GTA & GLA: Hardware-Efficient Attention That Breaks the ‘Memory-Dominated’ DecodeTL;DRGTA (key–value tying) and GLA …
Paper “Let’s put an OS on the GPU”: A proposal for a GPU multitasking OS layer for the LLM eraOne-line summary (TL;DR)In …
Paper Link DroidSpeak: Reducing Prefill Latency by 1.7–3.1× through Cross-LLM Prefix-KV ReuseTL;DRWhen multiple LLMs share the same …
![[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs](https://pbs.twimg.com/media/GdyLXO9W4AADox0.jpg)
Paper Link Marconi: Rethinking Prefix Caching for the Hybrid LLM EraTL;DRMarconi introduces a prefix-caching framework for hybrid LLM …
![[Paper Review] SGLang: Efficient Execution of Structured Language Model Programs](https://cdn.bytez.com/mobilePapers/v2/neurips/94872/images/20-0.png)
Paper Link SGLang & RadixAttention: How Execution Optimization for “LM Programs” Achieved a 6.4x SpeedupTL;DRBy combining a …
![[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://icml.cc/media/PosterPDFs/ICML%202024/32613.png)
Paper Link Structured State Space Duality: Unifying SSMs and Attention with Mamba-2 for 2–8× AccelerationTL;DRStructured State-Space Duality …
![[Paper Review] Inference-Time Hyper-Scaling with KV Cache Compression](https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.05345/gradient.png)
Link to Paper Dynamic Memory Sparsification (DMS): Making LLM Hyper-Scaling a Reality with 8× KV Cache CompressionOne-Line Summary …
![[Paper Review] Llama-Nemotron: Efficient Reasoning Models](https://cdn-thumbnails.huggingface.co/social-thumbnails/collections/nvidia/llama-nemotron-67d92346030a2691293f200b.png)
Paper Link Hydragen: The Secret Weapon for Decoding Large Batches with Shared Prefixes up to 32× FasterTL;DRBy decomposing the prefix and …
![[Paper Review] KIMI K2: OPEN AGENTIC INTELLIGENCE](https://github.com/MoonshotAI/Kimi-K2/raw/main/figures/kimi-logo.png){kind=logo}
Paper Link Kimi K2: An Open-Source LLM’s Leap Toward Agentic IntelligenceTL;DRWith a 3-stage pipeline consisting of MuonClip pretraining + …
Paper Link Qwen 3: The Evolution of a Giant MoE Language Model with Adjustable Reasoning DepthTL;DR (in one line)Qwen 3 couples a …
![[Paper Review] Massive Activations in Large Language Models](https://eric-mingjie.github.io/massive-activations/assets/main_teaser_final.png)
Paper Link Massive Activations, Hidden Biases: A Reinterpretation of Self-Attention’s Secrets TL;DRJust 4–10 extreme scalar values (×10,000) …
Paper Link Peri-LayerNorm: A Third Option Beyond Post-LN and Pre-LNTL;DRBy simply adding another LayerNorm right after the residual …
![[paper review] SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-bit Training](https://cdn-uploads.huggingface.co/production/uploads/66c0a08bac74db25de8427ec/Tb20E3IJSV6PjcD9Nkvfg.png)
[Paper Review] SageAttention 3 & SageBwd — FP4-Powered Inference and 8-bit TrainingPaper link: https://arxiv.org/abs/2505.11594v1 📝 …
![[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding](https://www.storagereview.com/wp-content/uploads/2025/07/image2-2-png-e1752234784623.webp)
Paper Link Helix Parallelism: Breaking the Latency-Throughput Wall of Ultra-Long LLM DecodingTL;DRHelix Parallelism schedules Attention and …
Paper Subgoal Curriculum + CoT Consistency: DeepSeek-Prover-V2 Reshapes Automated Theorem ProvingTL;DRDeepSeek-Prover-V2, built on the …
Paper Inference-Time Scaling: How DeepSeek-GRM Surpassed Giant ModelsOne-Line Summary (TL;DR)“27B model × 32 samples”—With only …
Paper CODE I/O: From Code I/O + Natural-Language CoT to General-Purpose Reasoning — Lifting 7B-30B LLMs by +2 Points on Average with Data …
Paper Native Sparse Attention (NSA) — 11× faster even at 64k tokens, accuracy intactOne-line summary (TL;DR)NSA combines a three-branch …
Paper Janus-Pro 7B: Dual-Encoder Multimodal LLM That Outsmarts Bigger ModelsOne-line summary (TL;DR)By fully separating the SigLIP …
Paper One-line Summary (TL;DR)DeepSeek-V3 is an open-source SOTA that combines Aux-loss-free Load-Balancing Bias, FP8 mixed-precision …
Paper One-line summary (TL;DR)JanusFlow achieves state-of-the-art performance in both image understanding and generation (FID 9.51 / GenEval …