When a Machine Learns Something Wrong
Has anyone ever taught you something that turned out to be not quite right? Maybe you learned to pronounce a word the wrong way, and then had to relearn it later. Learning wrong things happens to everyone — and it happens to machines too. Today we will find out how machines can learn the wrong lesson, why it matters, and how people can fix it.
Bad Examples Lead to Bad Learning
Remember: a machine learns from its examples. So if the examples have a problem, the machine will learn that problem too. Here is a simple case. A scientist wants to teach a machine to tell wolves from dogs in photos. They collect lots of wolf photos — but most of the wolf photos happen to have snow in the background. They collect lots of dog photos — mostly taken inside houses. The machine studies all the photos. But instead of learning what wolves look like, it accidentally learns: if there is snow in the background, say wolf. If there is no snow, say dog. Then someone shows it a photo of a dog playing in snow. The machine says: wolf! It learned the wrong thing — not because it was broken, but because the examples were misleading.
A machine learns what its examples show it — even if the examples show something misleading by accident. This is called bias. It is not magic or meanness. It is just the machine finding the wrong pattern.
Bias in machine learning is a word for when a machine has learned a skewed or unfair lesson because of problems in its training data. Here is another example. Imagine a machine trained to recommend jobs. If most of the training examples show nurses who are women and engineers who are men, the machine might start assuming women should be nurses and men should be engineers — even though that is not fair or true. The machine did not decide to be unfair. It just copied the pattern it saw in the examples. That is why the people who build learning machines have a very important responsibility: they must check their examples carefully and make sure they are fair and representative.
Complete the sentence about what happens when training examples are misleading.
The good news: wrong learning can be fixed! Once scientists notice that a machine learned the wrong thing, they can collect better examples — more balanced, more fair — and retrain the machine. Sometimes they can add new examples that correct the problem. This is just like how you fix a wrong habit. If you learned to hold your pencil in an uncomfortable way, a teacher can show you the better way and help you practice until the new habit takes over. Machines do not feel embarrassed about being corrected. They just update themselves and try again.
When a learning machine is used to make important decisions — like who gets a loan or whose face a camera recognizes — a wrong lesson can actually be unfair to real people. That is why checking for bias is a serious and important job in AI.
A machine trained on wolf photos mostly taken in snow learned to say 'wolf' whenever it saw snow. What went wrong?
How can scientists fix a machine that learned the wrong thing?
Spot the Sneaky Pattern
- You are going to set up a tricky learning situation and see what goes wrong.
- Collect 10 small objects. Put them in two groups of 5. Make sure Group A objects are all the same color and Group B objects are all a different color — but the groups are actually supposed to be sorted by shape, not color.
- Ask a helper to guess the rule after seeing only 3 examples. Did they guess color or shape?
- Now show them 3 more examples that break the color pattern. Does their guess change?
- Talk about it: how is a helper fooled by a misleading pattern? How is this like a machine learning the wrong thing?