Large Language Models
The Transformer
The general-purpose block the whole field is built on.
You can jump straight in, but this star assumes Attention and Tokenization. Not recommended as a first stop.
Four layers of depth
Each layer ends with a quiz. Finish layer 4 and you own this concept.
- L1IntuitionHow attention, an MLP, residual connections, and normalization combine into 'the block' that every LLM is just a stack of.6m
40 XP - L2MechanicsPre-LN vs post-LN placement, encoder/decoder vs decoder-only architectures, and how the pieces assemble into a full model.10m
70 XP - L3CodeAssemble a full GPT-style transformer block and forward pass from scratch in PyTorch, and verify parameter counts against a real model.15m
110 XP - L4FoundationsThe full parameter and FLOP accounting for a dense transformer, and why arithmetic intensity makes autoregressive decoding fundamentally different from training.13m
180 XP
Where this leads
33 stars in the atlas.