What Is AI · L4 Foundations

Formalizing 'learning' as function approximation and empirical risk minimization, plus a look at why scale changed everything.

1950s-80s
Symbolic AI era
expert systems, logic programming
1990s-2000s
Classical ML
SVMs, decision trees, hand-engineered features
2012
Deep learning breakout
AlexNet on ImageNet, GPU training
2017-now
Scale era
Transformers + massive data + compute

The shift toward statistical, then deep, then large-scale learning.

Formally, machine learning is the search for a function $f_{θ} : X \to Y$ , parameterized by $θ$ , that minimizes expected loss over the true (unknown) data distribution $D$ :