Practicum Deep Learning at AAU

This repo contains the slides and exercises from the Practicum Deep Learning course at the University of Klagenfurt, Austria.

For any question/issue, please contact me: Pierre TASSEL.

Table of Contents

The course focus on the basics of Deep Learning, solving machine learning problems using PyTorch.

Lecture slides

1. Preliminaries
   1. Introduction
   2. Preliminaries
   3. NDArray
   4. Pandas
   5. Linear Algebra
   6. AutoGrad
   7. Calculus (optional)
   8. Motivation: Stable Diffusion
2. Linear Networks
   1. Linear Regression
   2. Vectorization
   3. Linear Regression From Scratch
   4. Weights & Biases
   5. Pytorch NN Module
   6. Softmax Regression
   7. Torch.nn
3. Deep Neural Networks
   1. Model Interpretability with Captum
   2. Multi Layers Perceptron
   3. Regularization
   4. Visualize Neural Networks
   5. Advanced Autograd
4. Convolutional Neural Network
   1. Limits of MLP
   2. Convolution
   3. LeNet MNIST 3b. Gradio LeNet MNIST
   4. ImageNet and AlexNet
   5. VGG 5b. VGG19 Style Transfer
   6. Network in Network
   7. Google LeNet
   8. BatchNorm
   9. ResNet
5. Computer Vision
   1. Data Augmentation
   2. Learning Rate Schedulers
   3. Fine-Tuning
   4. Transposed Convolution
   5. AutoEncoder
   6. Denoising AutoEncoder
   7. Self-Supervision
6. Sequence 2 Sequence
   1. Recurrent Neural Network
   2. Gated Recurrent Unit
   3. Encoder-Decoder Architecture
   4. The Annotated Transformer

Lab

1. Tensor Manipulation
2. Linear Regression
3. Logistic Regression
4. MNIST with MLP
5. Cat vs Dog
6. CIFAR10 Pre-Training

Acknowledgments

The material for this course are built upon the following resources:

• Dive into Deep Learning
• NYU DEEP LEARNING SPRING 2020 course
• PyTorch official tutorial
• Deep Learning with PyTorch by Luca Antiga
• PyTorch documentation
• UvA Deep Learning Tutorials
• François Fleuret's Deep Learning Course
• DeepLearning.AI TensorFlow Developer Professional Certificate