Mini-SGLang: reading the scheduler of an LLM serving framework
Reading sgl-project/mini-sglang in eight parts, pinned to a single commit — from the process boundaries a request crosses to the radix cache and two-stream overlap.
8 pages
Reading sgl-project/mini-sglang in eight parts, pinned to a single commit — from the process boundaries a request crosses to the radix cache and two-stream overlap.
Reading the 1,500 lines of CUDA in jmaczan/tiny-vllm across seven parts — from weight loading and GPU buffers to a block-table KV cache and continuous batching.