Backpropagation
How a network works out, for every weight, which way to nudge it, using the chain rule applied backwards through a computation graph.
The question backprop answers
Training means nudging every weight to lower the loss (the number measuring how wrong the network is). To nudge a weight correctly you need its gradient: how much the loss changes if that weight changes a little. A network can have millions of weights, so we need a cheap way to get all the gradients.
Backpropagation (backprop) is that way. It is not a learning algorithm by itself. It computes gradients. A separate step, gradient descent, uses them.
1 of 7