DLite: History-free Speculative Decoding for Agentic AI
DLite explained through a candidate-verification example, cache ownership, and a careful comparison of acceptance length and decoding speed.
Posts
All posts on technology, daily life, and thoughts.
118 pages
3 /
20
FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention
An analysis of FlashMask-3’s Hopper kernels, mask localization, load balancing, communication overlap, and long-context training results and limitations.
Trajectory-Level Speculation with Semantic Verification for Tool-Using Agents
An analysis of SpecAgent, where a weaker model drafts tool calls and a stronger model verifies them, covering its design, experimental results, and limitations.
HLA-WM: Hybrid Linear Attention for Long-Horizon Video World Models
HLA-WM retrieves past chunks using camera geometry to restore scene memory. At 60 seconds, its history memory is about 12× smaller than Full KV and 6× larger than GDN,…
Contextual Correction of Parallel Drafts for Speculative Decoding
DCOCO reselects existing parallel draft candidates using contextual scores. This review distinguishes accepted length from actual throughput and examines correction…
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Fixed-point behavior connects KV sharing, state reuse, and distilled prefill in looped language models. We review the savings and accuracy trade-offs demonstrated on…