Which knob caused the mistake?
Last time: AI walks downhill in small steps to reduce the error. But with billions of knobs, how does it know which ones to turn, and by how much?
EXAMPLE
A bowl of noodle soup sent back
Picture a noodle soup kitchen with three cooks: one makes the broth, one cooks the noodles, one does the seasoning. A customer complains: the soup is too salty.
TRACING BACK
Trace back through each step
The head chef doesn't scold the whole kitchen. He traces back from the bowl to each step. The heavy-handed seasoner has to change a lot, the noodle cook barely changes at all.
BACKPROPAGATION
The mistake flows backward
A neural network does exactly that. The mistake at the output is passed back through each layer, from the last layer to the first. This is called backpropagation.
SHARING THE BLAME
More blame, bigger fix
Connections that caused more of the mistake get turned more, those that caused less get turned less. In a single backward pass, the machine knows how to fix every knob at once.
WHY IT MATTERS
The foundation of AI
Backpropagation plus gradient descent is the foundation of modern AI. The chatbots you use today were all trained this way.
PART 5
What about a new kind of test?
Learning this way, a machine can get so good it memorizes all the practice questions. But what happens with a question it's never seen? See you in part 5.
This article is based on the video Tracing mistakes back from the Mark học AI channel. Watch the video (in Vietnamese) to see the animations.