Deep Learning with S-Shaped Rectified Linear Activation Units
Deep Learning with S-Shaped Rectified Linear Activation Units
Rectified linear activation units are important components for state-of-the-art deep convolutional networks. In this paper, we propose a novel S-shaped rectifiedlinear activation unit (SReLU) to learn both convexand non-convex functions, imitating the multiple function forms given by the two fundamental laws, namely the Webner-Fechner law and the Stevens law, in …