Strip away the hype and every neural network layer does the same thing: it takes a list of numbers (a vector), multiplies it by a grid of learned numbers (a matrix), and produces a new list of numbers. Stack enough of these transformations and you get a language model.
The vocabulary you actually need
- Scalar: a single number, like a temperature reading.
- Vector: an ordered list of numbers — a word embedding, an RGB pixel, a model's hidden state.
- Matrix: a grid of numbers — a layer's learned weights, or a batch of vectors stacked together.
- Tensor: the generalization to any number of dimensions — what PyTorch actually stores everything as.
Two vectors being 'similar' — the entire basis of search, recommendation, and attention — is measured by the dot product: multiply corresponding entries and sum them. Vectors that point in a similar direction get a large dot product; unrelated ones get close to zero.