Deep Learning

Boston University - Fall 2022

Class is held in CGS 505 on Tuesday and Thursday 3:30-4:45pm

Course staff and office hours

Wednesday 4-5pm, MCS 200C: Prof. Iddo Drori

Thursday 11am-12pm, MCS 103: Teaching Fellow, Vitali Petsiuk

Grader, Yida Xin

Grader, Wenda Qin

Textbook

The Science of Deep Learning, Iddo Drori, Cambridge University Press, 2022

Enrolled students receive a free online version

First Day of Classes (Tuesday, September 6)

Part I: Foundations

Lecture 1 (Tuesday, September 6): Introduction

Lecture 2 (Thursday, September 8): Forward and Backpropagation

Lecture 3 (Tuesday, September 13): Optimization

Lecture 4 (Thursday, September 15): Regularization

Part II: Architectures

Lecture 5 (Tuesday, September 20): Convolutional neural networks (CNNs)

Lecture 6 (Thursday, September 22): Sequence models (RNNs, LSTM, GRU)

Lecture 7 (Tuesday, September 27): Graph neural networks (GNNs)

Lecture 8 (Thursday, September 29): Transformers

Part III: Reinforcement Learning

Lecture 9 (Tuesday, October 4): Markov decision processes

Lecture 10 (Thursday, October 6): Reinforcement learning

Lecture 11 (Thursday, October 13): Deep reinforcement learning

Part IV: Generative Models

Lecture 12 (Tuesday, October 18): Deep reinforcement learning

Lecture 13 (Thursday, October 20): Imperfect information games

Lecture 14 (Tuesday, October 25): Generative adversarial networks

Lecture 15 (Thursday, October 27): Variational autoencoders

Lecture 16 (Tuesday, November 1): Diffusion models

Part V: Meta Learning

Lecture 17 (Thursday, November 3): Automated machine learning

Lecture 18 (Tuesday, November 8): Multi-task and online learning

Lecture 19 (Thursday, November 10): Meta and transfer learning

Part VI

Lecture 20 (Tuesday, November 15): Winning the NeurIPS 2022 Neural MMO Challenge

Lecture 21 (Thursday, November 17): Deep learning for education

Lecture 22 (Tuesday, November 22): Deep learning for climate science

Thanksgiving recess, academic holiday, no classes (Wednesday, November 23 - Sunday, November 27)

Lecture 23 (Tuesday, November 29): Quantum computing

Lecture 24 (Thursday, December 1): Deep learning for quantum computing

Projects

Lecture 25 (Tuesday, December 6): Presentations

Lecture 26 (Thursday, December 8): Presentations

Last Day of Classes (Monday, December 12)

Exercises: quiz and programming homework

Exercise 1: Forward and Backpropagation

Exercise 2: Optimization

Exercise 3: CNN's

Exercise 4: RNN's

Exercise 5: GNN's

Exercise 6: RL

Exercise 7: GAN's

Exercise 8: VAE's

Exercise 9: Meta learning

Exercise 10: Transformers

Tutorials

Tutorial 1: PyTorch

Tutorial 2: Neural MMO Challenge

Tutorial 3: Keras

Tutorial 4: CNN's with TensorFlow

Tutorial 5: RNN's

Tutorial 6: dgl.ai, GNN library

Tutorial 7: RLlib, reinforcement learning library

Tutorial 8: huggingface.co, Transformers library, stability.ai

Tutorial 9: pyro.ai, probabilistic programming library

Tutorial 10: learn2learn.net, meta learning library