Miles v0.1: Production-Level Post-Training

Thu, 10 Sep 2026 00:00:00 +0900

Paper

Miles v0.1: A “verified, clean, and scalable” full-stack system for frontier RL post-training

TL;DR: Miles is a full-stack RL post-training framework that resolves the train-rollout mismatch at the system level — the mismatch that arises when rollout generation and training run on different engines, kernels, and precisions. It unifies SGLang-based rollout, a Megatron-LM/FSDP trainer, and three weight-synchronization transports (broadcast/P2P/disk-delta), while also guaranteeing token exactness (TITO) and expert-routing replay (R3). As an end-to-end case study, it trains GLM-5.2 744B-A40B on 64 GB300 GPUs with fully asynchronous agentic RL, achieving a median step time of 263 seconds (source: §9.2, Fig. 5).

Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training

Thu, 10 Sep 2026 00:00:00 +0900

Paper

Two Designs That Make Speculative Decoding Practical for Large-Scale, Long-Context RL Post-Training

TL;DR : Rollout generation dominates the wall-clock time of RL post-training. This work accelerates it with speculative decoding, and adds online draft co-training so the draft does not lag behind as the policy evolves. However, existing systems fail to handle the branch attention used by advanced drafts (unsupported by context parallelism) and target features scattered across stages (a pipeline parallelism problem). This paper proposes (1) a CP technique that decomposes branch attention into a causal main-sequence component and a rank-local branch and merges them, and (2) TapChannel, which delivers target features outside the pipeline schedule, achieving 1.16–1.88× end-to-end speedup from 8B to 122B (source: §3.3, Tab. 1).

Reinforcement Learning on Jaehun's Blog

Miles v0.1: Production-Level Post-Training

Miles v0.1: A “verified, clean, and scalable” full-stack system for frontier RL post-training

Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training

Two Designs That Make Speculative Decoding Practical for Large-Scale, Long-Context RL Post-Training