Dr. Heuna Kim Mathematics and Computer Science

Normalization Techniques in Deep Neural Networks

The note contains the concept of covariance shift and compares the following normalization techniques in deep learning.

  • batch normalization
  • dropout
  • layer normalization
  • weight normalization
  • skip connection

here is the handwritten note