VLLM

All posts under tag "VLLM"

2 posts total

Sorted by date

[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs

[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs

[Paper Review] Marconi: Prefix Caching for the Era of Hybrid LLMs

Paper Link Marconi: Rethinking Prefix Caching for the Hybrid LLM Era TL;DR Marconi introduces a prefix-caching framework for hybrid LLM …

October 08, 2025

2411.19379v3 Marconi Hybrid LLM Prefix Caching Inference Optimization FLOP-aware Scheduling SSM vLLM Serving Efficiency

[Paper Review] Llama-Nemotron: Efficient Reasoning Models

[Paper Review] Llama-Nemotron: Efficient Reasoning Models

[Paper Review] Llama-Nemotron: Efficient Reasoning Models

Paper Link Hydragen: The Secret Weapon for Decoding Large Batches with Shared Prefixes up to 32× Faster TL;DR By decomposing the prefix and …

2505.00949v4 Hydragen Prefix Caching Shared Prefix Decoding Efficient Inference Softmax Decomposition LLM Serving Attention Optimization FlashAttention vLLM Batch Inference Matrix-Matrix GEMM TensorCore Optimization