SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture
Paper SPECTRA: A Reconfigurable LLM Accelerator That Changes Its Hardware With Every Inference StageTL;DRSpeculative decoding oscillates …
12 min
Speculative Decoding
Hardware Architecture
Efficient Inference
LLM Inference
Inference Acceleration
Transformer