Post-training & Alignment
Reward Models
A model that scores answers so another model can chase the score.
You can jump straight in, but this star assumes RLHF. Not recommended as a first stop.
Four layers of depth
Each layer ends with a quiz. Finish layer 4 and you own this concept.
- L1IntuitionWhat a reward model is: a scorer trained to imitate human preference, not to answer questions.5m
40 XP - L2MechanicsHow preference data becomes a training signal: the Bradley-Terry model and the ranking loss.8m
70 XP - L3CodeImplement and train a Bradley-Terry reward model head-to-tail in PyTorch.12m
110 XP - L4FoundationsThe statistical model behind reward-model training and the effect of miscalibration on downstream RL.15m
180 XP
33 stars in the atlas.