# Purpose
Following along with the playlist created by Vizuara on Youtube (https://www.youtube.com/playlist?list=PLPTV0NXA_ZSj6tNyn_UadmUeU3Q3oR-hu).

The primary objective is to gain a foundational understanding of simple neural networks including forward propagation, activation layers, backward propagation, and gradient descent.

# Lectures
## Lectures 1-2
Neurons and layers with matrices and numpy.

## Lectures 3-6
Multiple layer neural networks with matrices and numpy.

ReLU and softmax activation functions.

## Lectures 7-11
Cross entropy loss and optimizing a single layer with gradient descent.

## Lecture 12
Backpropagation for a single neuron.

## Lectures 13-17
Backpropagation for neuron layers and activation layers.

## Lecture 18
Backpropagation on the cross entropy loss function.

## Lectures 19-21
Combined backpropagation for softmax and cross entropy loss.

Entire backpropagation pipeline for neural networks.

Entire forward pass pipeline for neural networks.

## Lecture 22
Gradient descent for entire neural network.

## Lectures 23-24
Learing rate decay in optimization.

Momentum in training neural networks.

## Lectures 25-26
Coding the ADAGRAD optimizer for neural networks.

Coding the RMSprop optimizer for neural networks.

## Lecture 27
Coding the ADAM optimizer for neural networks.

## Lectures 28-31
Neural network testing, generilization, and overfitting.

K-Fold cross validation.

L1/L2 regularization to avoid overfitting.

Dropout layers in neural networks.