Foundations of Graphical Models
Fall 2016
Columbia University

Course information

• Description and Syllabus
• Instructor: David M. Blei (office hours)
• Teaching Assistant: Maja Rudolph (OH Thursdays 4:00PM-5:00PM; Mudd 4th floor DSI)
• Teaching Assistant: Dustin Tran (OH Wednesdays 2:00PM-3:00PM; Mudd 4th floor DSI)
• Meeting: Tuesdays and Thursdays,10:10AM-11:25AM, 903 SSW
• Piazza site

Course materials

Below are the readings and lecture notes. When available, we include a link to the PDF of the readings. Otherwise, they are available outside of D. Blei's office (912 SSW). These topics may span multiple lectures in the class.

The lecture notes are works in progress. If you have comments about them or notice errors, please email david.blei@columbia.edu.

1. Introduction
   
   • Reading #1: Build, compute, critique, repeat: Data analysis with latent variable models (Blei, 2014)
   • Slides
2. A Quick Review of Probability
3. Basics of Graphical Models
   • Reading #2: "Conditional Independence and Factorization" in Introduction to Probabilistic Graphical Models (Jordan, 2003).
4. Elimination, Tree Propagation, and the Hidden Markov Model
   • Reading #3: "The Elimination Algorithm" in Introduction to Probabilistic Graphical Models (Jordan, 2003)
   • Reading #4: "Probability Propagation and Factor Graphs" in Introduction to Probabilistic Graphical Models (Jordan, 2003)
5. Models, Data, and Statistical Concepts
   • Reading #5: "Statistical Concepts" in Introduction to Probabilistic Graphical Models (Jordan, 2003).
   • Reading #5: Some issues in the foundations of statistics (Freedman, 1995)
   • Reading #5 (optional): Model-based machine learning (Bishop, 2013)
6. Bayesian Mixture Models and the Gibbs Sampler
   • Reading #6: Gibbs sampling for the uninitiated (Resnik and Hardisty, 2010)
   • Reading #6 (optional): Probabilistic inference using Markov chain Monte Carlo (Neal, 1993).
7. Exponential Families and Conjugate Priors
   • Reading #7: "The Exponential Family" (Bishop, 2006; Section 2.4)
8. Mixed-membership Models and Mean-Field Variational Inference
   • Reading #8 (choose one): Probabilistic topic models (Blei, 2012)
   • Reading #8 (choose one): Inference of population structure using multilocus genotype data (Pritchard et al., 2000)
   • Reading #9: Variational inference: A review for statisticians (Blei et al., 2016)
9. Matrix Factorization and Recommendation Systems
   • Reading #10: Matrix factorization techniques for recommender systems (Koren et al., 2009)
   • Reading #10 (optional): Bayesian probabilistic matrix factorization using Markov chain Monte Carlo (Salakhutdinov and Mnih, 2008)
10. Generalized Linear Models
    • Reading #11: "An outline of generalized linear models" in Generalized Linear Models (McCullough and Nelder, 1989)
    • Reading #11: "Multilevel linear structures" in Data Analysis Using Regression and Multilevel/Hierarchical Models (Gelman and Hill, 2007)
    • Reading #11: "Multilevel linear models: The basics" in Data Analysis Using Regression and Multilevel/Hierarchical Models (Gelman and Hill, 2007)

Homework

• Homework 1
• Homework 2
• Homework 3

Other materials

• LaTeX template
• Project guidelines