Normalization Techniques in Deep Neural Networks

March 20, 2017, Posted by Heuna Kim

The note contains the concept of covariance shift and compares the following normalization techniques in deep learning.

• batch normalization
• dropout
• layer normalization
• weight normalization
• skip connection

here is the handwritten note