1. The Processes One Request Passes Through
code-series
8 min
Archives
All posts by year.
74 pages
2026
43 pages
September 2026
42 pages
2. Req and Batch: the ledger for one request
code-series
8 min
3. The budget behind a prefill batch, and chunked prefill
code-series
7 min
4. Paged allocation, and a page table that holds raw locations
code-series
8 min
5. The radix cache that reuses prefixes, and eviction
code-series
9 min
6. Hiding CPU scheduling behind two streams
code-series
7 min
7. Where the ledger becomes kernel arguments, and CUDA graphs
code-series
8 min
8. How several ranks share one ledger
code-series
8 min
The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
paper-review, mixture-of-experts, with-muse-spark-1-3-contributor-free
18 min
1. An Inference Engine in 1,500 Lines of CUDA: The Map
code-series
10 min
2. Weight Loading and GPU Buffer Design
code-series
14 min
3. The prefill path: flowing the entire prompt at once
code-series
24 min
4. The cuBLAS Transposition Trick — Why the Matrices Are Passed Reversed
code-series
14 min
5. The decode path that produces one token — why separate kernels were made
code-series
16 min
StepAudio 3 Gen Technical Report
paper-review, model-architecture, with-deepseek-v4-pro
5 min
How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus
paper-review, speculative-decoding, with-deepseek-v4-pro
13 min
ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search
paper-review, long-context, with-deepseek-v4-pro
15 min
Programmable World Model
paper-review, model-architecture, with-deepseek-v4-pro
19 min
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
paper-review, kv-cache, with-deepseek-v4-pro
20 min
CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs
paper-review, sparsity-pruning, with-deepseek-v4-pro
4 min
Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
paper-review, model-architecture, with-deepseek-v4-pro
18 min
SQS: Bayesian DNN Compression through Sparse Quantized Sub-distributions
paper-review, sparsity-pruning, with-deepseek-v4-pro
13 min
Kalman Delta Networks: Uncertainty-aware Associative Memory
paper-review, model-architecture, with-deepseek-v4-pro
16 min
X-AuT: Progressive Audio-Encoder Compression for Speech LLMs with Cross-Scale Distillation
paper-review, sparsity-pruning, with-deepseek-v4-pro
11 min
Deadline-Aware Adaptive Prefill Chunking for Efficient Large Language Model Serving
paper-review, serving-systems, with-deepseek-v4-pro
15 min
Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs
paper-review, long-context, with-deepseek-v4-pro
17 min
Miles v0.1: Production-Level Post-Training
paper-review, training-efficiency, with-deepseek-v4-pro
15 min
Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training
paper-review, speculative-decoding, with-deepseek-v4-pro
11 min
Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference
paper-review, mixture-of-experts, with-muse-spark-1-3-contributor-free
20 min
VestigeKV: The NoPE-MLA KV Cache Carries Its Own Eviction Signal in a Vestigial Branch
paper-review, kv-cache, with-deepseek-v4-pro
15 min
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
paper-review, kv-cache, with-deepseek-v4-pro
17 min
Unlocking Lossless Speedups in LLMs via Discrete Diffusion
paper-review, speculative-decoding, with-deepseek-v4-pro
21 min
BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
paper-review, kv-cache, with-deepseek-v4-pro
14 min
Hardware-Aware FP4 FlashAttention-4
paper-review, quantization, with-deepseek-v4-pro
34 min
Language Models Can Control Their Own Attention
paper-review, kv-cache, with-deepseek-v4-pro
14 min
Same Request, Different Answer: Quantization Amplifies Cache-Induced Divergence in LLM Serving
paper-review, kv-cache, with-deepseek-v4-pro
16 min
SGD-KV: Summarization Guided KV Cache Compression
paper-review, kv-cache, with-deepseek-v4-pro
13 min
SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers
paper-review, mixture-of-experts, with-deepseek-v4-pro
11 min
Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM
paper-review, quantization, with-deepseek-v4-pro
14 min
Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
paper-review, kv-cache, with-qwen3-8-flash
21 min
January 2026
1 page
[Paper Review] Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models
paper-review, with-gpt, llm-architecture, systems-ml
39 min
2025
31 pages
December 2025
4 pages
[Paper Review] Continuous Autoregressive Language Models
paper-review, with-gpt-5.2
43 min
[Paper Review] Memory Retrieval and Consolidation in Large Language Models through Function Tokens
paper-review, interpretability, LLM Systems, with-gpt-5.2
23 min
[Paper Review] NVIDIA Nemotron 3: Efficient and Open Intelligence
paper-review, with-gpt-5.2
4 min
[Paper Review] Radial Attention: O(n log n) Sparse Attention with Energy Decay for Long Video Generation
paper-review, with-gpt-5.2
35 min
October 2025
8 pages
[Paper Review] Attention Sinks and Compression Valleys in LLMs are Two Sides of the Same Coin
paper-review, with-gpt
34 min
[Paper Review] Hardware-Efficient Attention for Fast Decoding
paper-review, with-gpt, Large Language Models, Model Efficiency, Compiler and Systems
35 min
[Paper Review] Towards Efficient and Practical GPU Multitasking in the Era of LLM
paper-review, with-gpt, LLM Systems, Model Serving and Scheduling, GPU Virtualization and Multi-Tenancy
36 min
[Paper Review] DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving
paper-review, LLM Serving and Systems, Model Optimization and Acceleration
15 min
[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs
paper-review, with-gpt, LLM Systems, Model Serving
11 min
[Paper Review] SGLang: Efficient Execution of Structured Language Model Programs
paper-review, with-gpt
39 min
[Paper Review] Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
paper-review, with-gpt
19 min
July 2025
19 pages
[Paper Review] Inference-Time Hyper-Scaling with KV Cache Compression
paper-review, with-gpt
32 min
[Paper Review] Llama-Nemotron: Efficient Reasoning Models
paper-review, with-gpt, efficient-llm, system-optimization, inference-acceleration
25 min
[Paper Review] KIMI K2: OPEN AGENTIC INTELLIGENCE
paper-review, with-gpt, open-source, agentic-intelligence, RL-alignment, foundation-models
14 min
[Paper Review] Qwen 3 Technical Report
paper-review, foundation-models, with-gpt
13 min
K-Beauty: Beyond 'Accidental Success' to 'Structural Growth'
Industry Analysis, Cosmetics Industry
4 min
A Guide to Migrating from Jekyll to Hugo
Daily
6 min
K-Beauty: On the Verge of a New Leap Forward?
Industry Analysis, Cosmetics Industry
9 min
Migrating My Blog from Jekyll to Hugo
Daily
3 min
[Paper Review] Massive Activations in Large Language Models
paper-review, with-gpt
21 min
[Paper Review] Peri-LN: Revisiting Normalization Layer in the Transformer Architecture
paper-review, with-gpt
24 min
[paper review] SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-bit Training
paper-review, with-gpt
23 min
[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding
paper-review, with-gpt
17 min
DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition
paper-review, with-gpt
26 min
Inference-Time Scaling for Generalist Reward Modeling
paper-review, with-gpt
25 min
Code I/O: Condensing Reasoning Patterns via Code Input-Output Prediction
paper-review, with-gpt
36 min
Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention
paper-review, with-gpt
36 min
Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling
paper-review, with-gpt, Janus, DeepSeek
18 min
DeepSeek-V3 Technical Report
paper-review, with-gpt
39 min
JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generation
paper-review, with-gpt
30 min