How a Network Practices
Nobody is great at something the very first time they try it. You probably wobbled the first time you rode a bike. You missed the ball the first time you tried to hit it. But you practiced, noticed your mistakes, and kept improving. Neural networks do the same thing — they practice, notice mistakes, and adjust. Let us find out how!
Making Mistakes on Purpose
To teach a neural network, we give it thousands of examples where we already know the right answer. For example, to teach a network to recognize dogs, we show it ten thousand photos labeled 'dog' or 'not dog.' The network looks at the first photo and guesses. Maybe it guesses wrong — it says 'not dog' but it was actually a dog! That mistake gets measured. The measurement of how wrong the guess was is called the loss. A big loss means the network was very wrong. A small loss means it was close.
Loss is the score for how wrong a network's guess was. A big loss means a big mistake. The network's goal during training is to make the loss as small as possible — to get really close to right every time.
Once the network knows how wrong it was, it needs to fix itself. It does this by working backward through all its layers — from the output back through the hidden layers all the way to the start. As it goes backward, it nudges each weight a tiny bit in whichever direction would have made the answer closer to correct. This backward walk is called backpropagation — a big word that just means 'send the mistake back so everyone can learn from it.' After nudging the weights, the network tries the next photo. It keeps doing this for all ten thousand photos. Then it loops through them all again. And again. After many rounds, its weights settle into values that let it recognize dogs very well.
Fill in the blank to complete the sentence about how networks learn.
One practice run through all the training examples is called an epoch. A network might go through fifty or a hundred epochs before it is well trained. That is like practicing your bike fifty times around the block. Each round, you get a little steadier. Each epoch, the network gets a little more accurate.
Training a large neural network can take hours, days, or even weeks — even on powerful computers. The good news: once it is trained, it can answer questions in a tiny fraction of a second.
What is 'loss' in neural network training?
What is backpropagation?
Practice Makes Perfect — Loss Experiment
- Get a small bucket or bowl and ten balled-up pieces of paper.
- Stand three steps away from the bucket. Toss all ten balls and count how many miss. That count is your loss.
- Write down your loss score.
- Now adjust — step a tiny bit closer, or change how hard you throw. Toss all ten again and count misses.
- Keep adjusting and recording your loss each round.
- Try five rounds and look at your numbers. Are they going down? You are doing exactly what a neural network does: practice, measure the mistake (loss), adjust, repeat!