from: http://en.wikipedia.org/wiki/Rectifier_(neural_networks)
In the context of artificial neural networks, the rectifier is an activation function defined as
Noisy ReLUs[edit]
Rectified linear units can be extended to include Gaussian noise, making them noisy ReLUs, giving
Noisy ReLUs have been used with some success in restricted Boltzmann machines for computer vision tasks.