Topics and readings
Below are topics of the class and some readings about each. (These
topics and readings are subject to change.)
The readings are at different levels: some are basic and some are
advanced. We chose them to provide fundamental and other interesting
material about the topics. Note that the lectures will not necessarily
cover all of this material.
The ingredients of probabilistic models
Linear and Logistic Regression
Stochastic Optimization
Bayesian mixture models and the Gibbs sampler
"Identifying Bayesian mixture models"
(Betancourt, 2018)
"Probabilistic inference using Markov chain Monte Carlo methods"
(Sections 1-4)
(Neal, 1993)
"The Collapsed Gibbs Sampler in Bayesian Computations with
Applications to a Gene Regulation Problem" (Liu, 1994)
Mixed-membership models, topic models, and variational inference
Matrix factorization and efficient MAP inference
Exponential families, conjugate priors, and generalized linear models
"The exponential family" (Bishop, 2006; Section 2.4)
"An outline of generalized linear models" (McCullagh
and Nelder, 1989; Chapter 2)
"Conjugate priors for exponential families"
(Diaconis and Ylvisaker, 1979)
"Exponential families in theory and practice" (Efron, 2018)
Hierarchical models, robust models, and empirical Bayes
"Multi-level structures" (Gelman and Hill, 2007; Chapter 11)
"Multi-level linear models: The basics" (Gelman and Hill, 2007;
Chapter 12)
"Bayes, oracle Bayes, and empirical Bayes" (Efron, 2017)
Deep probabilistic models
Advanced topics in variational inference
"Black box variational inference" (Ranganath et al.,
2014)
"Graphical models, exponential families, and variational
inference"
(Waingwright and Jordan, 2008)
"Monte Carlo gradient estimation in machine learning" (Mohamed e
al., 2019)
"Automatic differentation variational inference" (Kucukelbir et
al., 2017)
"Covariance, robustness, and variational Bayes"
(Broderick et al., 2018)
"ELBO surgery: Yet another way to carve up the variational
evidence lower bound" (Hoffman and Johnson,
2016)
"An optimization-centric view on Bayes' rule: Reviewing and
generalizing variational inference" (Knoblauch et al.,
2022)
The theory of graphical models
"Conditional independence and factorization" (Jordan, 2003;
Chapter 2)
"The elimination algorithm" (Jordan, 2003; Chapter 3)
"Probability propagation and factor graphs" (Jordan, 2003; Chapter
4)
Model criticism and model diagnosis
An introduction to causality