abseil
0Korean renderings of Abseil's Tip of the Week — string_view, unique_ptr, temporaries and lifetimes, and the other C++ questions that keep coming back.
Jaehun Ryu
A blog about technology, daily life, and thoughts.
I am a deep learning compiler engineer at Rebellions. This blog was established for the purpose of studying compiler technologies and optimization methods required to execute AI models, particularly LLMs, with greater efficiency on hardware. Furthermore, I intend to publish articles concerning my hobbies and investments intermittently.
Browse by topic
code-series
15Reading an open-source inference engine end to end, pinned to one commit. Each part follows the execution path through a specific region of the code and the reasoning behind its design.
LLM-Inference
0Paper reviews on making LLM inference faster and cheaper: speculative decoding, KV cache compression, prompt compression, sparsity, and heterogeneous pipelines.
KV Cache
25Paper reviews on shrinking, evicting and reusing the KV cache: compression, quantization, eviction signals, offloading and prefix reuse.
Long Context
20Paper reviews on coping with long context: sparse attention, context compression and recompute, state-space models, and million-token serving.
Distributed Computing
4Paper reviews on LLM serving and training across GPUs and nodes: prefill/decode disaggregation, pipeline parallelism, overlapping communication with compute, scheduling and energy.
Recent Posts
The KV Cache Is the New Memory Wall
Paper The KV Cache Is the New Memory Wall: Three Regimes and Five Domains of Long-Context LLM Inference TL;DR — Long-context LLM decode …
19 min
KV Cache
Efficient Inference
Long Context
Hardware Architecture
Attention Optimization
Quantization
WavePP: High-Throughput Pipeline Parallel LLM Prefill under Prefix Reuse
Paper WavePP — A Pipeline-Parallel Prefill Runtime That Burns In Prefix ReuseTL;DRBuilt on top of TensorRT-LLM, WavePP overlaps request …
18 min
KV Cache
Prefix Caching
Efficient Inference
LLM Inference
Pipeline Parallelism
Long Context
Periodic Weak Spots: Phase Sensitivity from Chunked KV-Cache Compression
Paper Phase Sensitivity: The ‘Periodic Weakness’ Created by Chunked KV-Cache Compression TL;DR — In models that use chunked …
19 min
KV Cache
Long Context
Large Language Models
Transformer
Interpretability
Efficient Inference
WorldAttention: An Efficient Attention Architecture for Interactive Video World Models
Paper WorldAttention: System Co-Design of an Interactive Video World Model That Drops the Sliding WindowTL;DR — Text-controlled interactive …
17 min
KV Cache
Efficient Inference
Long Context
Attention Optimization
Sparse Attention
Video Generation
Who Pays for the KV Cache? Attributing Shared AI Inference Spend Across Kubernetes and LLM Provider Bills
Paper Who pays for the KV cache? — Kubernetes, gateway, and provider bills in one ledger TL;DR — The cost of a single AI feature is …
14 min
KV Cache
Prefix Caching
LLM Inference
vLLM
Distributed Computing
Tensor Parallelism