Inference-Time Scaling for Generalist Reward Modeling

Tue, 08 Jul 2025 00:00:00 +0900

Paper

Inference-Time Scaling: How DeepSeek-GRM Surpassed Giant Models

One-Line Summary (TL;DR)

“27B model × 32 samples”—With only a Generative Reward Model (GRM) and k-Vote summation, it records 72.8% overall accuracy, higher than GPT-4o and Nemotron-340B, presenting inference compute instead of model size as a new scaling axis.

RLHF on Jaehun's Blog

Inference-Time Scaling for Generalist Reward Modeling

Inference-Time Scaling: How DeepSeek-GRM Surpassed Giant Models

One-Line Summary (TL;DR)