Harel Lidar

Paper summary · Aug 2026

Decoding Speculative Decoding

Summary of Decoding Speculative Decoding

Through 350 experiments the authors aim to delineate the factors that effect the performance of speculative decoding and observe that it relies heavily on the latency of the draft model, and that the draft model’s language modeling capabilities are not as strongly correlated with performance as one would thing. These insights lead to new design spaces for draft models which are more hardware efficient.