How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus
Paper Is “Lossless” Speculative Decoding Really Lossless? — The Numerical-Precision Trap Exposed by Reproducing OrthrusTL;DR — …
13 min
Speculative Decoding
Numerical Precision
Quantization
LLM Inference
Efficient Inference
Benchmark Evaluation