Meet machine learning: the big picture
Machine learning is getting a computer to learn a pattern from examples, instead of a person writing the rules by hand.
Why it matters
Some rules are too hard to write down. Try writing exact rules for "is this email spam?" or "what is this house worth?": there are too many cases and exceptions. But examples are easy to collect: thousands of emails already marked as spam or not, thousands of houses with their sale prices. Machine learning turns those examples into a working rule.
The idea
Machine learning means learning a pattern from examples instead of writing the rules by hand.
You give a model inputs (features) and the right answers (labels). The model makes a guess, a loss function measures how wrong it is, and an optimiser nudges the model's numbers (its weights) to make the loss smaller. Repeat that many times and the model learns.
Then you test it on data it has never seen, and if it's good, you deploy it.
Learning to throw darts. You throw (a prediction), see how far you landed from the bullseye (the loss), adjust your arm a little (an optimiser update), and throw again. After hundreds of throws, you're accurate.
A person can feel why a throw missed. A model only gets one number, the loss; the gradient then tells it which way to adjust every weight at once — sometimes millions of them.
How it works
- Collect examples. Each example has features (the inputs; here, the size in m²) and a label (the right answer; here, the price).
- Choose a model: a formula with adjustable numbers called weights. Here it is ŷ = w·size + b, a straight line. w and b start at 0, so the model knows nothing yet.
- Predict: run every example through the model. With w = 0, it predicts £0 for every house.
- Measure the loss: one number that says how wrong the predictions are. Mean squared error squares each error and takes the average. At the start it is 69,900.
- Adjust: the gradient of the loss says which way to move each weight to make the loss smaller. The optimiser moves each weight a small step that way, and the learning rate sets the size of the step. After one step, w = 4.66.
- Repeat predict → loss → adjust. w swings 4.66, 2.08, 3.51, 2.72… and each swing is smaller. After 1,000 loops w ≈ 3.0 and the loss is almost 0: each m² adds about £3,000.
- Test the model on data it never saw while training. Only if it does well there do you trust it and deploy it, so it can make predictions on new data in the real world.
Key points
- Feature: an input the model uses (size). Label: the right answer it learns to predict (price). One example is one row of both.
- Weights (also called parameters) are the numbers the model learns. Training means finding good weights.
- The loss is a single number for "how wrong". Training is nothing more than making that number smaller.
- Gradient descent: work out which way makes the loss smaller (the gradient), take a small step that way, and repeat.
- The learning rate is the step size. Too small and training is very slow; too big and the loss explodes.
- Always judge a model on data it was not trained on. Doing well on its own training data proves very little.
- Regression predicts a number (a price). Classification predicts a category (spam or not spam).
- This is supervised learning: every example has a label. Unsupervised learning finds structure without labels, such as grouping similar customers.
Common mistakes
- Thinking the model "understands" houses. It only found the numbers that make the loss small on these examples.
- Testing on the training data. A model can memorise its examples and still fail on new ones.
- Believing that more training always helps. After a point the model starts fitting noise; Unit 2 calls this overfitting.
- Mixing up the model and training. The model is the formula with weights; training is the loop that sets the weights.
Not true: A model that gets every training example right is certain to do well on new examples too.
In fact: A model can memorise its training examples and still fail on new ones. Only data it has never seen shows how good it is.
Not true: The longer a model trains, the better it gets on new data.
In fact: After a point, more training makes the model fit the noise in its examples, and it gets worse on new data. Unit 2 calls this overfitting.
Part of Unit 0: Your toolkit in the free ML Foundations course.