Determinants
The determinant of a square matrix is one number: how much the matrix scales area (or volume), with a minus sign if it flips space over.
Why it matters
You often need to know at a glance whether a matrix squashes space flat, because then it can't be undone and Ax = b has no unique answer. The determinant tells you: zero means squashed. It also appears whenever probabilities are moved through a transformation, as in the formula for a multivariate Gaussian and in normalising flows.
The idea
The determinant has two parts. Its size tells you how much the matrix scales area (or volume in 3D). Its sign tells you whether space gets flipped over.
Stretching a rubber sheet with a square drawn on it. The determinant says how much bigger the square becomes.
A rubber sheet cannot pass through itself. A negative determinant does exactly that: it is like seeing the sheet in a mirror.
How it works
- Start with the unit square: sides [1, 0] and [0, 1], area 1.
- Apply the matrix. The square becomes a parallelogram whose sides are the matrix's two columns.
- The determinant is that parallelogram's area, with a sign. For [[a, b], [c, d]] it is ad − bc.
- [[2, 0], [0, 3]] gives 2·3 − 0·0 = 6, so every area grows 6 times.
- A shear like [[1, 1], [0, 1]] gives 1·1 − 1·0 = 1. The square leans over but keeps its area.
- [[0, 1], [1, 0]] swaps x and y: 0·0 − 1·1 = −1. Same area, but space is flipped like a mirror image.
- [[2, 4], [1, 2]] gives 2·2 − 4·1 = 0. Both columns lie on one line, so the square is squashed flat, and the matrix has no inverse.
Key points
- |det| is how much areas (in 2-D) or volumes (in 3-D) are scaled.
- det < 0 means space is flipped; det = 0 means space is squashed into fewer dimensions.
- det = 0 ⇔ no inverse ⇔ the columns are dependent ⇔ the rank is not full.
- For a 2 × 2 matrix [[a, b], [c, d]], the determinant is ad − bc.
- det(AB) = det(A)·det(B): when you chain movements, their scale factors multiply.
- For big matrices use np.linalg.slogdet, which returns the log, because determinants can be astronomically large or small.
Common mistakes
- Using "the determinant is tiny" as a test for "nearly impossible to invert". 0.1 × the 100 × 100 identity has det 10⁻¹⁰⁰ but is trivial to invert. The condition number is the right measure.
- Treating a negative determinant as an error. It just means a flip.
- Asking for the determinant of a non-square matrix. It only exists for square ones.
Not true: |det| tiny (say 10⁻¹⁰⁰) = a matrix that is very hard to invert.
In fact: Not always: 0.1 times the 100 × 100 identity has a determinant of 10⁻¹⁰⁰, yet it is trivial to invert. The condition number is the right test.
Not true: Determinant of a 2 × 3 matrix: the product of its diagonal entries.
In fact: A determinant exists only for square matrices. A 2 × 3 matrix has none.
Part of Unit 1: Math foundations in the free ML Foundations course.