A brief history of deep learning and its success stories.
Perceptrons, Sigmoid neurons and Multi-Layer Perceptrons (MLP) with specific emphasis on their representation power and algorithms used for training them (such as Perceptron Learning Algorithm and Backpropagation).
Gradient Descent (GD) algorithm and its variants like Momentum based GD,AdaGrad, Adam etc
Principal Component Analysis and its relation to modern Autoencoders.
The bias variance tradeoff and regularisation techniques used in DNNs (such as L2 regularisation, noisy data augmentation, dropout, etc).
Different activation functions and weight initialization strategies
Convolutional Neural Networks (CNNs) such as AlexNet, ZFNet, VGGNet, InceptionNet and ResNet.
Recurrent Neural Network (RNNs) and their variants such as LSTMs and GRUs (in particular, understanding the vanishing/exploding gradient problem and how LSTMs overcome the vanishing gradient problem)
Applications of CNN and RNN models for various computer vision and Natural Language Processing (NLP) problems.