Large Language Models

Fine-tuning

Bending a general model toward your specific task.

Four layers of depth

Each layer ends with a quiz. Finish layer 4 and you own this concept.

1. L1IntuitionAdapting a pretrained model to a task or domain without retraining it from scratch — and when fine-tuning is the wrong tool entirely.7m
   40 XP
2. L2MechanicsBuilding a real SFT dataset with chat templates and loss masking, LoRA's mechanics, adapter hyperparameters, and forgetting/eval tradeoffs.10m
   70 XP
3. L3CodeA LoRA linear layer implemented from scratch, then a full PEFT + TRL SFT script with a chat template, masking, and evaluation.13m
   110 XP
4. L4FoundationsThe math of low-rank decomposition and its gradient scaling, QLoRA's NF4 quantization, and why fine-tuning hardware requirements differ so sharply from pretraining's.12m
   180 XP

Where this leads

33 stars in the atlas.