What Is Deep Learning · L2 Mechanics

The forward pass as a chain of transformations, why nonlinearity is essential, and the main deep architectures at a glance.

Input x
e.g. pixel vector
Linear transform
weighted sum
Nonlinearity
e.g. ReLU
Linear transform
next layer
Nonlinearity
...
Output
prediction

The forward pass: alternating linear transforms and nonlinear squashing functions.

Mechanically, a deep network is a chain of alternating steps: a linear transformation (multiply by weights, add a bias) followed by a nonlinear function applied element-wise. Repeat this pattern many times and you have a 'deep' network.