Quantization · L2 Mechanics

Scales and zero-points, symmetric vs asymmetric quantization, why outlier channels break naive schemes, and what GPTQ/AWQ/GGUF actually do differently.

Continuous weights
fp16, wide dynamic range
Choose scale + zero-point
per tensor, channel, or group
Round to nearest integer bucket
Store integers + scale
dequantize on the fly at inference

Every quantization scheme is a variant of this pipeline; the differences are all in step 2.

At its core, quantization maps a continuous range of real numbers onto a small fixed set of integers using a scale and a zero-point: $q = round (x / s) + z$ , and reconstructing an approximation later is $x^= s (q - z)$ . Symmetric quantization fixes $z = 0$ and centers the integer range on zero — simpler, and fine when weight distributions are roughly symmetric (which most are). Asymmetric quantization allows $z \neq = 0$ , useful for distributions that are lopsided (some activation distributions, especially after ReLU-family nonlinearities, are all non-negative).