Post-training & Alignment
RLHF
Learning from human preference instead of ground truth.
You can jump straight in, but this star assumes Fine-tuning. Not recommended as a first stop.
Four layers of depth
Each layer ends with a quiz. Finish layer 4 and you own this concept.
- L1IntuitionWhat RLHF is, why plain fine-tuning isn't enough, and the buzzwords decoded.5m
40 XP - L2MechanicsThe PPO training loop step by step: rollouts, reward scoring, KL penalty, and the policy update.8m
70 XP - L3CodeImplement the core pieces of a PPO update for language models: reward shaping and the clipped objective.12m
110 XP - L4FoundationsThe KL-constrained RL objective in full, why it has a closed-form optimal policy, and PPO's variance-reduction math.15m
180 XP
Where this leads
33 stars in the atlas.