DeepSeek on Jaehun's Blog

DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition

Tue, 08 Jul 2025 00:00:00 +0900

Paper

Subgoal Curriculum + CoT Consistency: DeepSeek-Prover-V2 Reshapes Automated Theorem Proving

TL;DR

DeepSeek-Prover-V2, built on the principle of “breaking a problem into small pieces and then matching them all the way through,” achieves a new SOTA on MiniF2F Pass@32 of 82.4% (671B) and 75.6% (7B) even in a small 7B model. The key is the combination of Subgoal-guided Curriculum and Chain-of-Thought-Lean consistency reward (GRPO).

Inference-Time Scaling for Generalist Reward Modeling

Tue, 08 Jul 2025 00:00:00 +0900

Paper

Inference-Time Scaling: How DeepSeek-GRM Surpassed Giant Models

One-Line Summary (TL;DR)

“27B model × 32 samples”—With only a Generative Reward Model (GRM) and k-Vote summation, it records 72.8% overall accuracy, higher than GPT-4o and Nemotron-340B, presenting inference compute instead of model size as a new scaling axis.

Code I/O: Condensing Reasoning Patterns via Code Input-Output Prediction

Mon, 07 Jul 2025 00:00:00 +0900

Paper

CODE I/O: From Code I/O + Natural-Language CoT to General-Purpose Reasoning — Lifting 7B-30B LLMs by +2 Points on Average with Data Design Alone

TL;DR

A single data pipeline of “code function → input/output prediction + structured Chain-of-Thought (CoT)” lets 3.5 M samples deliver larger and more balanced gains (+2.9 points) than 14 M-scale SOTA data. By nailing verifiability, low cost, and diversity at once, CODE I/O empirically demonstrates that “data quality > data quantity.”

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Mon, 07 Jul 2025 00:00:00 +0900

Paper

Native Sparse Attention (NSA) — 11× faster even at 64k tokens, accuracy intact

One-line summary (TL;DR)

NSA combines a three-branch sparse attention — “compress → select → slide” — with a GQA/MQA-friendly kernel to make decoding 11.6× faster and training up to 9× faster at 64k context, while improving average performance over Full Attention.

Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling

Sun, 06 Jul 2025 00:00:00 +0900

Paper

Janus-Pro 7B: Dual-Encoder Multimodal LLM That Outsmarts Bigger Models

One-line summary (TL;DR)

By fully separating the SigLIP understanding encoder from the VQ generation encoder and attaching them to a 7 B-parameter LLM with a ‘Dual-Encoder + Adapter’ design, it becomes the first case to beat a 13 B unified model (TokenFlow-XL) on both understanding and generation at the same time. – MMBench 79.2 (+10.3 pt) / GenEval 0.80 (+45 %)

DeepSeek-V3 Technical Report

Sat, 05 Jul 2025 00:00:00 +0900

Paper

One-line Summary (TL;DR)

DeepSeek-V3 is an open-source SOTA that combines Aux-loss-free Load-Balancing Bias, FP8 mixed-precision training, and Multi-Token Prediction in a 671B-parameter MoE LLM to reach parity with (or surpass) a dense 405B model at less than half the GPU time and cost.