Linear Algebra · L2 Mechanics

The mechanics of matrix multiplication, shapes, and why order and dimensions matter so much in practice.

Input vector (d,)
Weight matrix (d, k)
learned parameters
Matrix multiply
row · column dot products
Output vector (k,)

A linear layer, decomposed into the shapes that flow through it.

Matrix multiplication $C = A B$ is defined so that entry $C_{ij}$ is the dot product of row $i$ of $A$ and column $j$ of $B$ . This forces a shape rule: if $A$ is $(m, n)$ and $B$ is $(n, p)$ , the inner dimensions $n$ must match, and the result is $(m, p)$ .