Raw pixels
input
Layer 1: edges
learned automatically
Layer 2: shapes/textures
learned automatically
Layer 3: object parts
learned automatically
Output: 'cat'
prediction
Deep learning is machine learning using neural networks with many layers ('deep' = many layers stacked). Its defining advantage over classical ML is that it learns useful features automatically from raw data, instead of requiring a human to hand-engineer them.
The old way vs. the deep way
Classical ML pipeline
- Human designs features (edges, color histograms...)
- Feed features into a simple model (e.g. logistic regression)
- Ceiling limited by quality of hand-designed features
Deep learning pipeline
- Feed raw pixels/text/audio directly in
- Network learns its own hierarchy of features
- Ceiling scales with data and compute, not human insight
Why does stacking many simple layers work so well? Each layer transforms its input into a slightly more abstract representation. Early layers in an image model detect edges; middle layers combine edges into textures and shapes; late layers combine those into whole objects. No human decided any of this — it emerges from training on labeled examples.
Why it needed to wait for the 2010s
The core ideas behind deep learning are decades old. What changed was the availability of (1) large labeled/unlabeled datasets, (2) GPUs capable of the massive matrix multiplication these networks require, and (3) practical training tricks (better activations, initialization, normalization) that made very deep networks actually trainable instead of getting stuck.