Why does the gradient propagate in the "reverse" direction? Backpropagation can be understood using only the chain rule from high school mathematics. However, the concepts of differential geometry are ...
simple, easily readable, and easily modifiable. It is not optimized, and omits many desirable features.