Tokenization

Layer 2 · Mechanics

Tokenization

How BPE is actually trained and applied, byte-level fallback, special tokens, and vocabulary design trade-offs.

9 min read70 XP

  1. Pre-tokenize

    split on whitespace/punctuation boundaries

  2. Init vocab

    every byte is a starting token

  3. Count pairs

    frequency of every adjacent token pair

  4. Merge top pair

    add merged token to vocab

  5. Repeat

    until target vocab size reached

BPE training is a fixed, deterministic loop — no gradient descent involved.

Training a BPE tokenizer is entirely separate from training the language model, uses no gradients, and only needs to happen once. It runs over a large, representative text corpus and produces two artifacts: a vocabulary (a lookup table from token strings to integer IDs) and a merge list (the ordered sequence of merge rules learned, which is needed to tokenize new text the same way).

Go deeper: L3 Code