Deep Learning

Columbia University - Spring 2022

Class is held Tuesday and Thursday 1:10-2:25pm

Office hours (Monday-Friday)

Tuesday 2:30-3:30pm: Lecturer, Iddo Drori

Wednesday 3:30-4:30pm: Course Assistant, Anusha Misra

Friday 3-4pm: Course Assistant, Vaibhav Goyal

Thursday 3:30-4:30pm: Course Assistant, Chaewon Park

Monday 11am-12pm: Course Assistant, Vibhas Naik

Wednesday 10-11am: Course Assistant, Newman Cheng

Friday 11am-12pm: Course Assistant, Sri Thikkireddy

First Day of Classes (Tuesday, January 18)

Part I: Foundations

Lecture 1 (Tuesday, January 18): Introduction

Lecture 2 (Thursday, January 20): Forward and Backpropagation

Lecture 3 (Tuesday, January 25): Optimization

Lecture 4 (Thursday, January 27): Regularization

Part II: Architectures - CNNs, RNNs, GNNs, Transformers

Lecture 5 (Tuesday, February 1): Convolutional neural networks (CNNs)

Lecture 6 (Thursday, February 3): Sequence models (RNNs, LSTM, GRU)

Lecture 7 (Tuesday, February 8): Graph neural networks (GNNs)

Lecture 8 (Thursday, February 10): Transformers
Introduction, general purpose Transformer architectures, BERT
Self-attention. multi-head attention, Transformer, positional encoding, encoder, decoder, pre-training and fine-tuning
Transformer models: (i) auto-encoding Transformers, (ii) auto-regressive Transformers, (iii) sequence to sequence Transformers, GPT-3
Vision Transformers, multi-modal Transformers, text and code Transformers, OpenAI Codex.

Part III: Generative Models - GANs, VAEs, Normalizing Flows

Lecture 9 (Tuesday, February 15): Generative adversarial networks (GANs)

Lecture 10 (Thursday, February 17): Variational autoencoders (VAEs)

Lecture 11 (Tuesday, February 22): Normalizing flows

Part IV: Reinforcement Learning

Lecture 12 (Thursday, February 24): Reinforcement learning

Lecture 13 (Tuesday, March 1): Reinforcement learning

Lecture 14 (Thursday, March 3): Deep reinforcement learning

Lecture 15 (Tuesday, March 8): Deep reinforcement learning

Lecture 16 (Thursday, March 10): Deep reinforcement learning

Spring recess, academic holiday, no classes (Monday, March 14 - Friday, March 18)

Lecture 17 (Tuesday, March 22): Competition presentations

Part V: Meta Learning

Lecture 18 (Thursday, March 24): Automated machine learning

Lecture 19 (Tuesday, March 28): Multi-task learning

Lecture 20 (Thursday, March 31): Meta and transfer learning

Lecture 21 (Tuesday, April 5): Online and continual learning

Part VI: Applications

Lecture 22 (Thursday, April 7): Deep learning for proteomics

Lecture 23 (Tuesday, April 12): Deep learning for robotics

Lecture 24 (Thursday, April 14): Deep learning for space

Lecture 25 (Tuesday, April 19): Deep learning for climate science

Lecture 26 (Thursday, April 21): Deep learning for quantum computing

Lecture 27 (Tuesday, April 26): Deep learning for quantum computing

Projects

Lecture 28 (Thursday, April 28): Posters session

Last Day of Classes (Monday, May 2)

Exercises: quiz and programming homework

Exercise 1: Forward and Backpropagation

Exercise 2: Optimization

Exercise 3: CNN's

Exercise 4: RNN's

Exercise 5: GNN's

Exercise 6: Transformers

Exercise 7: GAN's

Exercise 8: VAE's

Exercise 9: Meta learning

Exercise 10: RL

Tutorials

Tutorial 1: PyTorch

Tutorial 2: TensorFlow

Tutorial 3: Keras

Tutorial 4: CNN's with TensorFlow

Tutorial 5: RNN's

Tutorial 6: dgl.ai, GNN library

Tutorial 7: huggingface.co, Transformers library

Tutorial 8: pyro.ai, probabilistic programming library

Tutorial 9: learn2learn.net, meta learning library

Tutorial 10: RLlib, reinforcement learning library