Posts

All posts on technology, daily life, and thoughts.

74 pages

12 / 13

[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding

[Paper Review] Helix Parallelism: Rethinking Sharding Strategies for Interactive Multi-Million-Token LLM Decoding

Paper Link Helix Parallelism: Breaking the Latency-Throughput Wall of Ultra-Long LLM DecodingTL;DRHelix Parallelism schedules Attention and …

Tensor Parallelism Mixture of Experts FlashAttention Long Context Inference

DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition

Paper Subgoal Curriculum + CoT Consistency: DeepSeek-Prover-V2 Reshapes Automated Theorem ProvingTL;DRDeepSeek-Prover-V2, built on the …

math-llm Chain-of-Thought proof-verification formal-mathematics DeepSeek

Inference-Time Scaling for Generalist Reward Modeling

Paper Inference-Time Scaling: How DeepSeek-GRM Surpassed Giant ModelsOne-Line Summary (TL;DR)“27B model × 32 samples”—With only …

Generative Reward Model LLM Evaluation Preference Modeling RLHF DeepSeek

Code I/O: Condensing Reasoning Patterns via Code Input-Output Prediction

Paper CODE I/O: From Code I/O + Natural-Language CoT to General-Purpose Reasoning — Lifting 7B-30B LLMs by +2 Points on Average with Data …

DeepSeek LLM Chain-of-Thought Instruction Tuning Transformer Long Context

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Paper Native Sparse Attention (NSA) — 11× faster even at 64k tokens, accuracy intactOne-line summary (TL;DR)NSA combines a three-branch …

Long Context Transformer Optimization GPU Acceleration FlashAttention Memory Efficiency DeepSeek

Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling

Paper Janus-Pro 7B: Dual-Encoder Multimodal LLM That Outsmarts Bigger ModelsOne-line summary (TL;DR)By fully separating the SigLIP …

DeepSeek Dual Encoder Multimodal Learning Vision-Language Models Large Language Models Efficient Training