Foundations of Graphical Models
Fall 2014
Columbia University

Course information

• Description and Syllabus
• Instructor: David M. Blei
• Teaching Assistant: Kui Tang
• Meeting: Mondays and Wednesdays, 1:10PM-2:25PM, Hamilton Hall 703
• Office hours for Blei: Wednesdays, 2:30PM-4:30PM, CEPSR 703
• Communicate and discuss on Piazza .

Lecture Notes

• Introduction   [Slides] [Notes]
• Graphical Models   [Notes]
• Elimination, Sum Product (and the HMM)   [Notes]
• Statistical Models   [Notes]
• Markov Chain Monte Carlo (and Bayesian Mixture Models)   [Notes]
• The Exponential Family and Conjugate Priors   [Notes]
• Mixed-Membership Models   [Notes]
• Linear Regression, Logistic Regression, Generalized Linear Models   [Notes]
• Hierarchical Linear Models   [Notes]
• Regularized Linear Models   [Notes]

Readings

1. D. Blei.   Build, compute, critique, repeat: Data analysis with latent variable models.   Annual Review of Statistics and Its Application 1:203-232, 2014.   [Link]
2. M. Jordan.   Conditional independence and factorization.   An Introduction to Probabilistic Graphical Models, 2003.
3. M. Jordan.   The elimination algorithm.   An Introduction to Probabilistic Graphical Models, 2003.
4. M. Jordan.   Probability propagation and factor graphs.   An Introduction to Probabilistic Graphical Models, 2003.
5. D. Freedman.   Some issues in the foundations of statistics.   Foundations of Science, 1:19-39, 1994.   [PDF]
6. R. Neal.   Probabilistic inference using Markov chain Monte Carlo. University of Toronto Department Technical Report, CRG-TR-93-1, 1993.   [PDF]
7. D. Blei.   Probabilistic topic models.   Communications of the ACM, 55(4):77–84, 2012.   [PDF]
8. J. Pritchard, M. Stephens, and P. Donnelly.   Inference of population structure using multilocus genotype data. Genetics, 155:945–959, June 2000. [PDF]
9. P. McCullagh and J. Nelder.   An outline of generalized linear models. In Generalized Linear Models, 1989.
10. A. Gelman and J. Hill.   Multilevel structures. In Applied Regression and Multilevel/Hierarchical Models, 2007.
11. A. Gelman and J. Hill.   Multilevel linear models: The basics. In Applied Regression and Multilevel/Hierarchical Models, 2007.
12. B. Efron.   Empirical Bayes and the James-Stein estimator. In Large-Scale Inference, 2010. [PDF]
13. T. Hastie, R. Tibshirani, and J. Friedman.   The Elements of Statistical Learning, 2nd Edition. Springer, February 2009. [Link]
14. M. Hoffman, D. Blei, J. Paisley, and C. Wang.   Stochastic variational inference.   Journal of Machine Learning Research, 14:1303-1347, 2013.   [PDF]

Other materials

• Reading response latex template