Deep Learning Batch Normalization note
Batch Normalization (BN)
� Normalizing input (LeCun et al 1998 �Efficient Backprop�)
� BN: normalizing each layer, for each mini-batch
� Greatly accelerate training
� Less sensitive to initialization
� Improve regularization
S. Ioffe & C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML 2015
Deep Residual Learning for image recognition, presents that do batch normalization before active function will be better.
