SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture

Sat, 26 Sep 2026 00:00:00 +0900

Paper

SPECTRA: A Reconfigurable LLM Accelerator That Changes Its Hardware With Every Inference Stage

TL;DR

Speculative decoding oscillates between prefill (compute-bound GEMM), decode (memory-bound GEMV), and the parallel verification in between, so arithmetic intensity fluctuates from token to token. A fixed datapath cannot handle all three regimes at once. SPECTRA is a reconfigurable architecture that switches the PE array inside a tile between systolic and vector-lane per kernel, and dynamically selects the tile count, sharding axis, and communication primitives across tiles. On a 20-tile FPGA prototype it achieves up to 2.09× over a fixed design (tile level) plus an additional 1.25× (system level), delivering up to 3.6× throughput and 2.9× power efficiency over edge GPUs.

Hardware-Accelerators on Jaehun's Blog

SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture

SPECTRA: A Reconfigurable LLM Accelerator That Changes Its Hardware With Every Inference Stage

TL;DR