All posts on technology, daily life, and thoughts.
Peri-LayerNorm: A Third Option Beyond Post-LN and Pre-LN TL;DR By simply adding another LayerNorm right after the residual connection—"Peri-LayerNorm (Peri-LN)"—the…
![[paper review] SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-bit Training](/assets/images/papers/2505.11594/figure-2.png)
Separate SageAttention3 FP4 inference from SageBwd INT8 training, with checked examples of quantization error, storage overhead and kernel versus whole-workload speed.
![[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding](https://www.storagereview.com/wp-content/uploads/2025/07/image2-2-png-e1752234784623.webp)
Helix Parallelism: Breaking the Latency-Throughput Wall of Ultra-Long LLM Decoding TL;DR Helix Parallelism schedules Attention and FFN with different parallelism…
Subgoal Curriculum + CoT Consistency: DeepSeek-Prover-V2 Reshapes Automated Theorem Proving TL;DR DeepSeek-Prover-V2, built on the principle of "breaking a problem into…
Inference-Time Scaling: How DeepSeek-GRM Surpassed Giant Models One-Line Summary (TL;DR) "27B model × 32 samples"—With only a Generative Reward Model (GRM) and k-Vote…
CODE I/O: From Code I/O + Natural-Language CoT to General-Purpose Reasoning — Lifting 7B-30B LLMs by +2 Points on Average with Data Design Alone TL;DR A single data…