Deep Learning

Boston University - Fall 2023

Class is held in CAS 203 on Tuesday and Thursday 3:30-4:45pm

Course staff and office hours

Instructor: Prof. Iddo Drori, Thursday 2-3pm, CDS 839

Teaching Fellow: Greeshma Yaluru, TBD

Grader: Pranesh Jayasundar

Grader: Nishant Vijay Nadkarni

Textbook

The Science of Deep Learning, Iddo Drori, Cambridge University Press, 2022

Enrolled students receive a free online version

First Day of Classes (Tuesday, September 5)

Part I: Foundations

Lecture 1 (Tuesday, September 5): Introduction

Lecture 2 (Thursday, September 7): Forward and Backpropagation

Lecture 3 (Tuesday, September 12): Optimization

Lecture 4 (Thursday, September 14): Regularization

Part II: Architectures

Lecture 5 (Tuesday, September 19): Convolutional neural networks (CNNs)

Lecture 6 (Thursday, September 21): Sequence models (RNNs, LSTM, GRU)

Lecture 7 (Tuesday, September 26): Graph neural networks (GNNs)

Lecture 8 (Thursday, September 28): Transformers

Part III: Reinforcement Learning

Lecture 9 (Tuesday, October 3): Markov decision processes

Lecture 10 (Thursday, October 5): Reinforcement learning

Lecture 11 (Tuesday, October 10): Deep reinforcement learning

Lecture 12 (Thursday, October 12): Deep reinforcement learning

Lecture 13 (Tuesday, October 17): Imperfect information games

Part IV: Generative Models

Lecture 14 (Thursday, October 19): Generative adversarial networks

Lecture 15 (Tuesday, October 24): Variational autoencoders

Lecture 16 (Thursday, October 26): Diffusion models

Part V: Meta Learning

Lecture 17 (Tuesday, October 31): Automated machine learning

Lecture 18 (Thursday, November 2): Multi-task and online learning

Lecture 19 (Tuesday, November 7): Meta and transfer learning

Part VI

Lecture 20 (Thursday, November 9): Ethics in deep learning

Lecture 21 (Thursday, November 14): Deep learning for education

Lecture 22 (Thursday, November 16): Deep learning for climate science

Lecture 22 (Thursday, November 21): Deep learning for quantum error correction

Thanksgiving recess, academic holiday, no classes (Wednesday, November 22 - Sunday, November 26)

Part VII

Lecture 23 (Tuesday, November 28): Common task summary

Lecture 24 (Thursday, November 30): TBD

Lecture 25 (Tuesday, December 5): Presentations

Lecture 26 (Thursday, December 7): Presentations

Lecture 27 (Tuesday, December 12): Summary

Last Day of Classes (Tuesday, December 12)

Exercises: quiz and programming homework

Exercise 1: Forward and Backpropagation

Exercise 2: Optimization

Exercise 3: CNNs

Exercise 4: RNNs

Exercise 5: LLMs

Exercise 6: LLMs

Exercise 7: GNNs

Exercise 8: Diffusion models

Exercise 9: Meta learning

Exercise 10: TBD

Tutorials

Tutorial 1: PyTorch

Tutorial 2: Common task framework

Tutorial 3: Keras

Tutorial 4: LLM APIs

Tutorial 5: LLM fine-tuning

Tutorial 6: GNN's

Tutorial 7: Reinforcement learning library

Tutorial 8: Meta learning

Tutorial 9: TBD

Tutorial 10: TBD