On the importance of normalisation layers in deep learning with piecewise linear activation units
On the importance of normalisation layers in deep learning with piecewise linear activation units
Deep feedforward neural networks with piecewise linear activations are currently producing the state-of-the-art results in several public datasets (e.g., CIFAR-10, CIFAR-100, MNIST, and SVHN). The combination of deep learning models and piecewise linear activation functions allows for the estimation of exponentially complex functions with the use of a large number …