Batch Normalization (BN) is a technique in deep learning that normalizes the inputs of a neural network layer by adjusting and scaling them. It was introduced to address issues related to internal covariate shift and has become a standard component in many modern neural network architectures. Here’s an in-depth look at Batch Normalization:

# 1. Internal Covariate Shift:

## Definition:

- Internal covariate shift refers to the change in the distribution of the input to a neural network layer during training.

## Issue:

- The shift in input distributions can slow down the training process as each layer has to continuously adapt to the changing inputs.

# 2. Batch Normalization Concept:

## Normalization:

- Batch Normalization normalizes the inputs of a layer by subtracting the mean and dividing by the standard deviation.

## Scale and Shift:

- The normalized values are then scaled and shifted using learnable parameters to allow the network to adapt during training.

# 3. Batch Normalization Procedure:

## For a Mini-Batch:

- Given a mini-batch of activations, calculate the mean and standard deviation for each feature.
- Normalize the features using the mean and…