Unit 1: Math foundations
The language of ML. You will see matrices as movements of space, gradients as directions, and loss functions as probabilities.
The big picture
Maths is the grammar of ML. You can memorise sentences (formulas), but you only speak fluently when you know the grammar. Linear algebra describes the data. Calculus describes how things change. Probability describes how sure you are.
Lessons in this unit
- Vectors, span and basis: A vector is a list of numbers that you can draw as an arrow. The span of some vectors is every point you can reach by scaling and adding them, and a basis is the smallest set that reaches the whole space.
- Matrices as transformations; matrix multiplication: A matrix is a recipe for moving every vector in space at once; multiplying two matrices means doing one movement after the other.
- Determinants: The determinant of a square matrix is one number: how much the matrix scales area (or volume), with a minus sign if it flips space over.
- Inverse, rank and solving Ax = b: The inverse of a matrix undoes what the matrix does. The rank counts how many independent directions survive it, and when the rank is full, Ax = b has exactly one solution.
- Dot product, projection and orthogonality: The dot product multiplies two vectors entry by entry and adds the results. It measures how much they point the same way, and it gives the length of one vector's shadow on another.
- Eigenvalues and eigenvectors: An eigenvector of a matrix is a direction the matrix does not turn, only stretches. Its eigenvalue is how much it is stretched.
- SVD, and the maths behind PCA: SVD splits any matrix into three simple moves: rotate, stretch, rotate. PCA uses it to find the directions in which data varies most, so you can keep a few and drop the rest.
- Derivatives and the chain rule: A derivative is the rate at which an output changes when you nudge the input; the chain rule says that for a function inside a function, you multiply the rates.
- Partial derivatives, gradients and Jacobians: A partial derivative is the rate of change when you nudge one input and hold the others still. The gradient puts all of them into one vector, which points in the direction of steepest increase.
- Gradient descent and the learning rate: Gradient descent makes a function smaller by taking many small steps downhill, against its gradient. The learning rate sets the size of each step.
- Random variables, distributions, expectation and variance: A random variable is a number decided by chance; its distribution says how likely each value is, its expectation is the long-run average, and its variance measures how spread out it is.
- Conditional probability and Bayes’ rule: Conditional probability is the chance of A once you know that B happened. Bayes' rule flips it round: from the chance of B given A, it gives the chance of A given B, using how common A was to begin with.
- Maximum likelihood estimation: Maximum likelihood estimation chooses the parameter values that make the data you actually observed as probable as possible.
- Statistics: sampling, confidence intervals, hypothesis tests: Statistics measures how far a result from a sample can be trusted. A confidence interval gives a likely range for the true value, and a hypothesis test asks whether a difference could just be luck.
- Entropy, cross-entropy and KL divergence: Entropy measures how uncertain a distribution is. Cross-entropy measures how surprised a model is by the real data, and KL divergence is the extra surprise that comes from the model being wrong.
- Convex optimisation: constraints, Lagrange multipliers and duality: A convex function is bowl-shaped, so any local minimum is the global one. A problem with a constraint is solved by adding the constraint with a Lagrange multiplier, and duality looks at the same problem from the constraint's side.