Measuring Error
In the last lesson you saw how a model turns input features into a prediction. But here is the obvious next question: what if the prediction is wrong? Every model starts out making bad predictions. The only way to get better is to know exactly how wrong it was — and that requires a precise measurement of error. This lesson is about building that measuring stick.
Prediction vs. Truth
When a model makes a prediction, we can compare it to the correct answer — the label or value that a human (or sensor, or record) confirmed is true. That correct answer is called the ground truth. Example: a model predicts that a house will sell for $320,000. The house actually sells for $350,000. The model was off by $30,000. That gap between prediction and truth is the error for that one example. For a classification model, error is simpler: either the model predicted the right class or it did not. If it predicted 'cat' and the photo was of a dog, that is one wrong — one error.
Ground truth is the confirmed correct answer for a training or test example. Error (also called loss for a single example) is the difference between the model's prediction and the ground truth. Smaller error = better prediction.
One error for one example does not tell us much. We need to measure error across many examples and combine them into a single score. That combined score is called the loss (or cost) function output — usually just called the loss. For regression problems, a common loss is Mean Squared Error (MSE). Here is how it works: 1. For each example, compute (prediction − ground truth). 2. Square that difference (so negative errors do not cancel positive ones). 3. Average all those squared differences. If a model predicts [10, 20, 30] for three examples and the true values are [12, 18, 33], the errors are [−2, 2, −3]. Squared: [4, 4, 9]. Average: 17/3 ≈ 5.67. That is the MSE — one number summarizing how far off the model was across all three examples.
Loss Functions for Classification
For classification problems, squaring differences does not make as much sense. Instead, the most common loss is cross-entropy loss, which measures how surprised the model should be by the true answer. Think of it this way: if the model says 'I am 99% sure this is a cat' and it really is a cat, the loss is tiny — that was a confident correct answer. If the model says 'I am 99% sure this is a cat' and it is actually a dog, the loss is enormous — the model was confidently wrong. Cross-entropy punishes confident wrong answers harshly. You do not need to calculate cross-entropy by hand right now. What matters: every model needs a loss function that translates 'how wrong was I?' into a single number the learning algorithm can use.
Loss is a continuous number used internally by the learning algorithm. Accuracy (percentage of correct answers) is what humans usually report. They are related but not the same. A model's loss can decrease while its accuracy temporarily stays flat. Do not confuse them.
Flashcards — click each card to reveal the answer
A regression model predicts 50 for an example whose true value is 40. What is the squared error for that single example?
Why does cross-entropy loss spike when a model is 99% confident but wrong?
Human Error Calculator
- Step 1: Make up five 'prediction vs. truth' pairs for a regression problem of your choice (e.g., predicting quiz scores: predicted [80, 70, 90, 60, 85], actual [75, 72, 88, 65, 80]).
- Step 2: For each pair, compute the error (predicted − actual) and then square it.
- Step 3: Average the five squared errors to get the MSE.
- Step 4: Change one prediction to be way off (e.g., predict 30 when truth is 75) and recalculate. Notice how one bad prediction can dramatically raise the MSE.
- Step 5: Write a one-sentence explanation of why squaring errors is useful.