Foundations of Graphical Models
Fall 2015
Columbia University

Course information

• Description and Syllabus
• Instructor: David M. Blei
• Teaching Assistant: Maja Rudolph
• Meeting: Mondays and Wednesdays, 1:10PM-2:25PM, 603 Hamilton
• Office hours for D. Blei: Wednesdays, 2:30PM-4:30PM, CEPSR 703
• Office hours for M. Rudolph: 3:30PM-4:30PM, Tuesdays (620 CEPSR) and Thursdays (912 SSW)
• Piazza site

Course materials

Below are the readings and lecture notes. When available, we include a link to the PDF of the readings. Otherwise, they are available outside of Prof. Blei's office. The specific reading assignments are announced on Piazza.

These topics may span multiple lectures in the class. See the syllabus for the schedule. Note that these lecture notes are drafts and works in progress. Feel free to email david.blei@columbia.edu with comments and errors.

1. Introduction
   
   • Reading: Build, compute, critique, repeat: Data analysis with latent variable models (Blei, 2014)
   • Slides
2. A Quick Review of Probability
3. Basics of Graphical Models
   • Reading: "Conditional Independence and Factorization" in Introduction to Probabilistic Graphical Models (Jordan, 2003).
4. Elimination, Tree Propagation, and the Hidden Markov Model
   • Reading: "The Elimination Algorithm" in Introduction to Probabilistic Graphical Models (Jordan, 2003)
   • Reading: "Probability Propagation and Factor Graphs" in Introduction to Probabilistic Graphical Models (Jordan, 2003)
5. Models, data, and statistical concepts
   • Reading: "Statistical Concepts" in Introduction to Probabilistic Graphical Models (Jordan, 2003).
   • Reading: Some issues in the foundations of statistics (Freedman, 1995)
   • Optional reading: Model-based machine learning (Bishop, 2013)
6. Bayesian Mixture Models and the Gibbs Sampler
   • Reading: Gibbs sampling for the uninitiated (Resnik and Hardisty, 2010)
7. Probabilistic Modeling in Stan
   • Reading: Stan Reference Manual 2.8.0 [Chs 1, 5.1, 9] (Stan Development Team, 2015)
8. Exponential Families and Conjugate Priors
   • Reading: "The Exponential Family" (Bishop, 2006; Section 2.4)
9. Mixed-membership Models and Mean-Field Variational Inference
   • Reading: Probabilistic topic models (Blei, 2012)
   • Reading: Inference of population structure using multilocus genotype data (Pritchard et al., 2000)
10. Matrix Factorization and Recommendation Systems
    • Reading: Matrix factorization techniques for recommender systems (Koren et al., 2009)
    • Optional reading: Bayesian probabilistic matrix factorization using Markov chain Monte Carlo (Salakhutdinov and Mnih, 2008)
11. Generalized Linear Models
    • Reading: "An outline of generalized linear models" in Generalized Linear Models (McCullough and Nelder, 1989)
    • Reading: "Multilevel linear structures" in Data Analysis Using Regression and Multilevel/Hierarchical Models (Gelman and Hill, 2007)
    • Reading: "Multilevel linear models: The basics" in Data Analysis Using Regression and Multilevel/Hierarchical Models (Gelman and Hill, 2007)
12. Regularized Regression
    • Reading: The Elements of Statistical Learning, Chapters 3.1, 3.2, 3.4, 3.6 (Hastie et al., 2009)
13. Bayesian Nonparametric Models
    • Reading: A tutorial on Bayesian nonparametric models" (Gershman and Blei, 2012)

Homework assignments

1. Homework 1
   Out: 2015-09-30
   Due: 2015-10-12
   

2. Homework 2
   Out: 2015-10-20
   Due: 2015-11-04
   

Other materials

• LaTeX template
• Project guidelines