Foundations of Graphical Models
Fall 2014
Columbia University
Course information
Lecture Notes
(These are drafts, and works in progress. Feel free to email david.blei@columbia.edu with comments and errors.)
- Introduction [Slides] [Notes]
- Graphical Models
[Notes]
- Elimination, Sum Product (and the HMM)
[Notes]
- Statistical Models
[Notes]
- Markov Chain Monte Carlo (and Bayesian Mixture Models)
[Notes]
- The Exponential Family and Conjugate Priors
[Notes]
- Mixed-Membership Models
[Notes]
- Linear Regression, Logistic Regression, Generalized Linear Models
[Notes]
- Hierarchical Linear Models
[Notes]
- Regularized Linear Models
[Notes]
Readings
When not available electronically, readings can be found outside
Prof. Blei's door.
- D. Blei. Build, compute, critique, repeat: Data
analysis with latent variable models. Annual Review of
Statistics and Its Application 1:203-232, 2014.
[Link]
- M. Jordan. Conditional independence and
factorization. An Introduction to Probabilistic Graphical
Models, 2003.
- M. Jordan. The elimination algorithm. An
Introduction to Probabilistic Graphical Models, 2003.
- M. Jordan. Probability propagation and factor
graphs. An Introduction to Probabilistic Graphical
Models, 2003.
- D. Freedman. Some issues in the foundations of
statistics. Foundations of Science, 1:19-39,
1994. [PDF]
- R. Neal. Probabilistic inference using Markov chain
Monte Carlo. University of Toronto Department Technical
Report, CRG-TR-93-1, 1993.
[PDF]
- D. Blei. Probabilistic topic models.
Communications of the ACM, 55(4):77–84, 2012.
[PDF]
- J. Pritchard, M. Stephens, and P. Donnelly. Inference
of population structure using multilocus genotype
data. Genetics, 155:945–959, June
2000. [PDF]
- P. McCullagh and J. Nelder. An outline of generalized
linear models. In Generalized Linear Models, 1989.
- A. Gelman and J. Hill. Multilevel structures.
In Applied Regression and Multilevel/Hierarchical Models,
2007.
- A. Gelman and J. Hill. Multilevel linear models: The
basics. In Applied Regression and Multilevel/Hierarchical
Models, 2007.
- B. Efron. Empirical Bayes and the James-Stein
estimator. In Large-Scale Inference,
2010. [PDF]
- T. Hastie, R. Tibshirani, and J. Friedman. The Elements
of Statistical Learning, 2nd Edition. Springer, February 2009.
[Link]
- M. Hoffman, D. Blei, J. Paisley, and C. Wang.
Stochastic variational inference. Journal of Machine
Learning Research, 14:1303-1347, 2013.
[PDF]
Other materials