Three ReLU networks with different widths learning noisy step(x)

Three ReLU networks with different widths learning noisy sin

Three ReLU networks with different widths learning sign(sin⁡(x))

What is ReLU?

ReLU Isn’t Always the Best Choice

Swish vs ReLU — Why Swish Works So Well

Three ReLU networks with different widths learning noisy parabola

Dying ReLU problem and variants of ReLU #deeplearning #machinelearning #relu #ai #ml #neuralnetworks

ReLU Fixed One Problem… Created Another 😬