Sinusoidal absolute encoding
The original Transformer paper (Vaswani et al., 2017) assigns each position a vector built from sine and cosine waves of different frequencies, one pair of dimensions per frequency band. Low-numbered dimensions oscillate fast (fine-grained position), high-numbered ones oscillate slowly (coarse-grained position) — the same idea as a clock with a second hand and an hour hand encoding time at two resolutions simultaneously. This vector is added elementwise to the token embedding before the first layer.