ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding

Tue, 08 Sep 2026 00:00:00 +0900

Paper

ShallowStream: Index Shallowly, Answer Deeply — Unlocking the Computational Bottleneck of Streaming Video Understanding

TL;DR — In streaming video understanding, the most expensive cost is prefilling every frame through the MLLM’s full depth (28–32 layers). ShallowStream builds frame encoding and a retrieval index with only 4–5 shallow layers, and runs full-depth computation on the retrieved evidence only at the moment a question arrives. As a result, it keeps SOTA-level performance (OVO-Bench 69.5 / StreamingBench 78.2) while cutting per-frame prefill by up to 52.1× and 10-second end-to-end latency by up to 11.9× (source: Abstract).

2609.02780v1 on Jaehun's Blog

ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding

ShallowStream: Index Shallowly, Answer Deeply — Unlocking the Computational Bottleneck of Streaming Video Understanding