The affine quantization map, precisely
For a real value in range mapped to -bit unsigned integers :
Layer 4 · Foundations
Quantization
The scale/zero-point math in full, the variance of rounding error, and why decoding is memory-bandwidth bound — the real reason quantization speeds things up.
10 min read180 XP