TAO: Tool-Aware Joint Scheduling for LLM-Based Agentic Serving
paper-review, serving-systems, with-deepseek-v4-pro
30 min
Archives
All posts by year.
118 pages
2026
87 pages
November 2026
1 page
October 2026
29 pages
Look Only When Needed: Certificate-Guided Visual Routing for Efficient Diffusion VLMs
paper-review, with-deepseek-v4-pro
15 min
On the SPOT: Enhancing Speculative Decoding for Diffusion Language Models via Commit-Order Prediction
paper-review, speculative-decoding, with-deepseek-v4-pro
23 min
Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data
paper-review, mixture-of-experts, with-deepseek-v4-pro
19 min
Extreme Compression of Per-Layer Embeddings via Cross-Layer Latent Vector Quantization
paper-review, quantization, with-deepseek-v4-pro
15 min
Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation
paper-review, model-architecture, with-deepseek-v4-pro
24 min
Matrix-Game 4.0: Harnessing Video Models for Real-Time Playable Game Worlds
paper-review, model-architecture, with-deepseek-v4-pro
17 min
Rethinking Sequence Compression with Test-time Regression
paper-review, long-context, with-deepseek-v4-pro
18 min
Large Flow Language Models
paper-review, speculative-decoding, with-deepseek-v4-pro
19 min
Cache Engineering for Agents
paper-review, kv-cache, with-deepseek-v4-pro
17 min
Speculative Sampling in Interactive Video Generation
paper-review, serving-systems, with-deepseek-v4-pro
22 min
MemT-1.0: Towards Memory-Native Evolution
paper-review, with-deepseek-v4-pro
17 min
DLite: History-free Speculative Decoding for Agentic AI
paper-review, speculative-decoding, with-deepseek-v4-pro
10 min
FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention
paper-review, attention-kernels, with-deepseek-v4-pro
13 min
Trajectory-Level Speculation with Semantic Verification for Tool-Using Agents
paper-review, speculative-decoding, with-deepseek-v4-1-flash-free
35 min
HLA-WM: Hybrid Linear Attention for Long-Horizon Video World Models
paper-review, long-context, with-deepseek-v4-pro
17 min
Contextual Correction of Parallel Drafts for Speculative Decoding
paper-review, speculative-decoding, with-deepseek-v4-pro
21 min
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
paper-review, kv-cache, with-deepseek-v4-pro
17 min
SparseDecoding: Decoding-Aware Pruning for Accurate and Efficient LLM Inference
paper-review, sparsity-pruning, with-deepseek-v4-pro
16 min
MC-Sparse: Deconstructing and Closing the Dense-Sparse Attention Gap in Diffusion Transformers
paper-review, attention-kernels, with-deepseek-v4-pro
15 min
TokenRouter: Efficient Serving System for Token-Level LLM Routing
paper-review, serving-systems, with-deepseek-v4-pro
20 min
Mechanics of Long-Context Hybrid Models Part 1.1: From Hybrid Attention to Hybrid Position
paper-review, long-context, with-deepseek-v4-pro
33 min
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
paper-review, quantization, with-deepseek-v4-pro
11 min
SlimKV: Joint Token-Feature KV Cache Compression with Reconstruction-Free Beacon Attention
paper-review, kv-cache, with-deepseek-v4-pro
15 min
Tailoring the Quantization Space for 1-Bit KV Cache Compression
paper-review, kv-cache, with-deepseek-v4-pro
16 min
Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
paper-review, kv-cache, with-muse-spark-1-3-contributor-free
20 min
On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics
paper-review, training-efficiency, with-deepseek-v4-pro
16 min
The KV Cache Is the New Memory Wall
paper-review, kv-cache, with-deepseek-v4-pro
19 min
WavePP: High-Throughput Pipeline Parallel LLM Prefill under Prefix Reuse
paper-review, serving-systems, with-deepseek-v4-pro
18 min
Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
paper-review, kv-cache, with-deepseek-v4-pro
19 min
September 2026
56 pages
MassAlloc Attention: Let Attention Allocate Its Own Compute
paper-review, attention-kernels, with-deepseek-v4-pro
21 min
WorldAttention: An Efficient Attention Architecture for Interactive Video World Models
paper-review, kv-cache, with-deepseek-v4-pro
17 min
Block Sparse Attention with Log-Linear Complexity
paper-review, long-context, with-deepseek-v4-pro
18 min
Who Pays for the KV Cache? Attributing Shared AI Inference Spend Across Kubernetes and LLM Provider Bills
paper-review, serving-systems, with-deepseek-v4-pro
14 min
GDN Tree-Scan: Served Tree Verification for Recurrent-Hybrid Language Models
paper-review, speculative-decoding, with-deepseek-v4-pro
17 min
SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture
paper-review, hardware-accelerators, with-deepseek-v4-pro
12 min
Adapting Tree-Structured Speculative Decoding to DeepSeek-V4 for Efficient Inference
paper-review, speculative-decoding, with-muse-spark-1-3-contributor-free
14 min
Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
paper-review, kv-cache, with-muse-spark-1-3-contributor-free
19 min
H-Spec: Parallel Speculative Decoding Without a Drafter-Side KV Cache
paper-review, speculative-decoding, with-muse-spark-1-3-contributor-free
17 min
Analytical Power-Aware Provisioning for Prefill-Decode Disaggregated AI Inference
paper-review, serving-systems, with-muse-spark-1-3-contributor-free
14 min
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
paper-review, attention-kernels, with-muse-spark-1-3-contributor-free
18 min
ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models
paper-review, quantization, with-deepseek-v4-pro
20 min
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
paper-review, kv-cache, with-deepseek-v4-pro
16 min
1. The Processes One Request Passes Through
code-series
8 min
2. Req and Batch: the ledger for one request
code-series
8 min
3. The budget behind a prefill batch, and chunked prefill
code-series
7 min
4. Paged allocation, and a page table that holds raw locations
code-series
8 min
5. The radix cache that reuses prefixes, and eviction
code-series
9 min
6. Hiding CPU scheduling behind two streams
code-series
7 min
7. Where the ledger becomes kernel arguments, and CUDA graphs
code-series
8 min
8. How several ranks share one ledger
code-series
8 min
Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches
paper-review, kv-cache, with-muse-spark-1-3-contributor-free
22 min
The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
paper-review, mixture-of-experts, with-muse-spark-1-3-contributor-free
18 min
1. An Inference Engine in 1,500 Lines of CUDA: The Map
code-series
10 min
2. Weight Loading and GPU Buffer Design
code-series
14 min
3. The prefill path: flowing the entire prompt at once
code-series
24 min
4. The cuBLAS Transposition Trick — Why the Matrices Are Passed Reversed
code-series
14 min
5. The decode path that produces one token — why separate kernels were made
code-series
16 min
StepAudio 3 Gen Technical Report
paper-review, model-architecture, with-deepseek-v4-pro
18 min
How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus
paper-review, speculative-decoding, with-deepseek-v4-pro
13 min
ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search
paper-review, long-context, with-deepseek-v4-pro
15 min
Programmable World Model
paper-review, model-architecture, with-deepseek-v4-pro
19 min
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
paper-review, kv-cache, with-deepseek-v4-pro
20 min
CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs
paper-review, sparsity-pruning, with-deepseek-v4-pro
15 min
Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
paper-review, model-architecture, with-deepseek-v4-pro
18 min
SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
paper-review, sparsity-pruning, with-deepseek-v4-pro
13 min
Kalman Delta Networks: Uncertainty-aware Associative Memory
paper-review, model-architecture, with-deepseek-v4-pro
16 min
X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation
paper-review, sparsity-pruning, with-deepseek-v4-pro
15 min
Deadline-Aware Adaptive Prefill Chunking for Efficient Large Language Model Serving
paper-review, serving-systems, with-deepseek-v4-pro
15 min
Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs
paper-review, long-context, with-deepseek-v4-pro
17 min
Miles v0.1: Production-Level Post-Training
paper-review, training-efficiency, with-deepseek-v4-pro
15 min
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
paper-review, speculative-decoding, with-deepseek-v4-pro
11 min
Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference
paper-review, mixture-of-experts, with-muse-spark-1-3-contributor-free
20 min
VestigeKV: The NoPE-MLA KV Cache Carries Its Own Eviction Signal in a Vestigial Branch
paper-review, kv-cache, with-deepseek-v4-pro
15 min
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
paper-review, kv-cache, with-deepseek-v4-pro
17 min
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
paper-review, speculative-decoding, with-deepseek-v4-pro
21 min
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
paper-review, kv-cache, with-deepseek-v4-pro
14 min
Hardware-Aware FP4 FlashAttention-4
paper-review, quantization, with-deepseek-v4-pro
34 min
Language Models Can Control Their Own Attention
paper-review, kv-cache, with-deepseek-v4-pro
14 min
Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving
paper-review, kv-cache, with-deepseek-v4-pro
16 min
SGD-KV: Summarization Guided KV Cache Compression
paper-review, kv-cache, with-deepseek-v4-pro
13 min
SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
paper-review, mixture-of-experts, with-deepseek-v4-pro
11 min
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
paper-review, quantization, with-deepseek-v4-pro
14 min
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
paper-review, kv-cache, with-qwen3-8-flash
21 min
January 2026
1 page
[Paper Review] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
paper-review, with-gpt, llm-architecture, systems-ml
39 min
2025
31 pages
December 2025
4 pages
[Paper Review] Continuous Autoregressive Language Models
paper-review, with-gpt-5.2
43 min
[Paper Review] Memory Retrieval and Consolidation in Large Language Models through Function Tokens
paper-review, interpretability, LLM Systems, with-gpt-5.2
23 min
[Paper Review] NVIDIA Nemotron 3: Efficient and Open Intelligence
paper-review, with-gpt-5.2
43 min
[Paper Review] Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation
paper-review, with-gpt-5.2
35 min
October 2025
8 pages
[Paper Review] Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
paper-review, with-gpt
34 min
[Paper Review] Hardware-Efficient Attention for Fast Decoding
paper-review, with-gpt, Large Language Models, Model Efficiency, Compiler and Systems
35 min
[Paper Review] Towards Efficient and Practical GPU Multitasking in the Era of LLM
paper-review, with-gpt, LLM Systems, Model Serving and Scheduling, GPU Virtualization and Multi-Tenancy
36 min
[Paper Review] DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving
paper-review, LLM Serving and Systems, Model Optimization and Acceleration
15 min
[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs
paper-review, with-gpt, LLM Systems, Model Serving
11 min
[Paper Review] SGLang: Efficient Execution of Structured Language Model Programs
paper-review, with-gpt
39 min
[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
paper-review, with-gpt
19 min
July 2025
19 pages
[Paper Review] Inference-Time Hyper-Scaling with KV Cache Compression
paper-review, with-gpt
32 min
[Paper Review] Llama-Nemotron: Efficient Reasoning Models
paper-review, with-gpt, efficient-llm, system-optimization, inference-acceleration
25 min
[Paper Review] KIMI K2: OPEN AGENTIC INTELLIGENCE
paper-review, with-gpt, open-source, agentic-intelligence, RL-alignment, foundation-models
14 min
[Paper Review] Qwen 3 Technical Report
paper-review, foundation-models, with-gpt
13 min
K-Beauty: Beyond 'Accidental Success' to 'Structural Growth'
Industry Analysis, Cosmetics Industry
4 min
A Guide to Migrating from Jekyll to Hugo
Daily
6 min
K-Beauty: On the Verge of a New Leap Forward?
Industry Analysis, Cosmetics Industry
9 min
Migrating My Blog from Jekyll to Hugo
Daily
3 min
[Paper Review] Massive Activations in Large Language Models
paper-review, with-gpt
21 min
[Paper Review] Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
paper-review, with-gpt
24 min
[paper review] SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-bit Training
paper-review, with-gpt
2 min
[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding
paper-review, with-gpt
17 min
DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
paper-review, with-gpt
26 min
Inference-Time Scaling for Generalist Reward Modeling
paper-review, with-gpt
25 min
Code I/O: Condensing Reasoning Patterns via Code Input-Output Prediction
paper-review, with-gpt
36 min
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
paper-review, with-gpt
36 min
Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
paper-review, with-gpt, Janus, DeepSeek
18 min
DeepSeek-V3 Technical Report
paper-review, with-gpt
39 min
JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation
paper-review, with-gpt
30 min