Large Language Models
Tokenization
Turning text into the integers a model can eat.
You can jump straight in, but this star assumes Embeddings. Not recommended as a first stop.
Four layers of depth
Each layer ends with a quiz. Finish layer 4 and you own this concept.
- L1IntuitionWhy a model can't read text at all — it reads integers — and how those integers get chosen.6m
40 XP - L2MechanicsHow BPE is actually trained and applied, byte-level fallback, special tokens, and vocabulary design trade-offs.9m
70 XP - L3CodeBuild a working byte-level BPE trainer and encoder from scratch, then compare it to a real tokenizer.14m
110 XP - L4FoundationsThe formal objective BPE greedily approximates, entropy and compression bounds, and the hardware cost of vocabulary size.12m
180 XP
Where this leads
33 stars in the atlas.