The Training Loop

Every machine learning system you have ever encountered — whether it labels photos, suggests videos, or generates text — was built through the same fundamental engine: a loop that makes a guess, measures how wrong the guess was, adjusts something, and then tries again. Repeat millions of times. This is the training loop, and understanding it reveals what learning actually means for a machine.

The Four Steps

The training loop has four steps that repeat over and over until the model becomes good enough. Step 1 — Predict: The model sees an input (say, an image) and produces an output (say, cat). At the start of training the model has random parameters and its prediction is essentially a random guess. Step 2 — Compare: The model's prediction is compared to the correct answer from the training label. The difference is measured by a function called the loss (sometimes called the error or cost). A big loss means a bad prediction. A loss of zero means a perfect prediction. Step 3 — Adjust: A mathematical procedure called backpropagation calculates which parameters were most responsible for the error. Then an update rule — typically gradient descent — nudges each parameter in the direction that reduces the loss. Step 4 — Repeat: The loop starts again with the next training example (or the next batch of examples). Over millions of iterations the parameters edge closer and closer to values that produce accurate predictions.

Here is a concrete analogy. Imagine you are learning to throw a basketball. You shoot (predict). You see whether it went in (compare). You adjust your stance and release angle based on what felt wrong (adjust). You shoot again (repeat). Over hundreds of shots, your body's parameters — muscle memory — converge toward the right form. The training loop is exactly this process, formalized in mathematics.

Loss: The Machine's Signal of Wrong

The loss function is the numerical measure of how far off the model's prediction was. Choosing the right loss function matters enormously. For a yes/no classification task the loss might measure whether the model assigned high probability to the wrong class. For a regression task — predicting a house price — the loss might measure the squared difference between the predicted price and the actual sale price. A well-designed loss function tells the model exactly how to be wrong in a way that is useful for getting better. A poorly chosen loss function can cause the model to optimize for something slightly different from what you actually care about — a subtle and important failure mode.