TokenRouter: Efficient Serving System for Token-Level LLM Routing
Paper TokenRouter: An Efficient Serving System for Token-Level LLM Routing TL;DR — Routing an LLM at the token level rather than the …
20 min
Efficient Inference
LLM Inference
Large Language Models
Mixture of Experts
Transformer
Model Routing