Matrices as transformations; matrix multiplication

A matrix is a recipe for moving every vector in space at once; multiplying two matrices means doing one movement after the other.

Why it matters

A neural network layer, a rotation of an image, a change of units, squeezing data down to fewer features: all of these are "do the same linear thing to every vector". A matrix stores that movement in a small grid of numbers, and matrix multiplication lets a computer apply it to millions of vectors, or chain several movements, very fast.

The idea

A matrix is a machine that moves every vector in space. Its columns show where the basis vectors land. Multiplying two matrices means doing one movement after the other.

Instructions for a photo editor: rotate, then stretch.

In the analogy the order feels unimportant, but it matters. Rotate-then-stretch is usually different from stretch-then-rotate, so AB ≠ BA.

How it works

  1. Watch where the two basis vectors land. A = [[2, 0], [0, 3]] sends [1, 0] to [2, 0], its first column, and [0, 1] to [0, 3], its second column.
  2. Every other vector follows, because it is a combination of the basis vectors: [1, 1] = [1, 0] + [0, 1] lands on [2, 0] + [0, 3] = [2, 3].
  3. That is matrix-times-vector: Av = v₁·(first column) + v₂·(second column). Grid lines stay straight, parallel and evenly spaced, and the origin stays put.
  4. To do B first and then A, multiply AB, read right to left. Each column of AB is A applied to the matching column of B.
  5. Order matters. With R a 90° rotation, A·R (rotate, then stretch) is [[0, −2], [3, 0]], but R·A (stretch, then rotate) is [[0, −3], [2, 0]].
  6. Shapes must fit: an (m, n) matrix times an (n, p) matrix gives an (m, p) matrix. The two inner sizes must match.

Key points

Common mistakes

Not true: In NumPy, A * B computes the matrix product of A and B.
In fact: A * B multiplies the entries in the same position. The matrix product is written A @ B.

Not true: For two matrices, AB always equals BA.
In fact: The order matters. AB means do B first, then A, and swapping them usually gives a different result.

Part of Unit 1: Math foundations in the free ML Foundations course.