Inference Optimization
Speculative Decoding
A small model guesses, a big model checks. Free speed.
You can jump straight in, but this star assumes The KV Cache. Not recommended as a first stop.
Four layers of depth
Each layer ends with a quiz. Finish layer 4 and you own this concept.
- L1IntuitionHow a small, fast model can speed up a big, slow model's generation without changing its outputs.5m
40 XP - L2MechanicsThe accept/reject mechanics that make speculative decoding exactly lossless.8m
70 XP - L3CodeImplement the speculative decoding accept/reject loop in PyTorch-style pseudocode.12m
110 XP - L4FoundationsProving speculative decoding is exactly lossless, and modeling its expected speedup.15m
180 XP
33 stars in the atlas.