COMS 4771 Fall 2023

COMS 4771 is a graduate-level introduction to machine learning. The course covers basic statistical principles of supervised machine learning, as well as some common algorithmic paradigms.

Course information

Announcements

Announcements will be posted on Ed Discussion.

Exams

Information about Exam 1 is available.

Information about Exam 2 is available.

Homework

Wondering how to prepare or submit your homework write-up? Please see the page on homework instructions and policies. Homeworks and accompanying data files are posted in Courseworks under “Files”.

Lecture schedule

Past and near-term lectures are listed here, along with links to lecture slides, readings, and other materials. Topics of other planned lectures can be found in the course syllabus. Lecture recordings are available on Courseworks.

Overview of machine learning (9/5)
slides, 2up
Dietterich overview article (through section 3)
(optional) Breiman’s “Two Cultures” article
Nearest neighbors (9/5, 9/7)
slides, 2up
(optional) [ESL] 2.3, 7.10, 13.3; [PC] 4.5
Classification using generative models (9/12)
slides, 2up
(optional) [ESL] 4.3; [PC] 2.1-2.6
Statistical models for prediction (9/14, 9/19)
slides, 2up
Error rate confidence intervals based on CLT approximation
Some uses of the binomial distribution
(optional) [ESL] 2.4; [PC] 2.3
Decision tree learning (9/21)
slides, 2up
(optional) [ESL] 9.2, 8.7; [PC] 8.2-8.4, 9.4.2, 9.5.1
Linear regression (9/26, 9/28)
slides, 2up
(optional) [ESL] 2.3.1, 3.1-3.2
Linear classification (10/3, 10/5)
slides, 2up
Notes on linear separators
(optional) Excerpt from Calvino’s “If on a winter’s night a traveler”
(optional) [ESL] 4.4, 4.5
Feature maps and kernels (10/10, 10/12)
slides, 2up
Hardt’s and Recht’s chapter on “Representations and features”
(optional) Freund’s and Schapire’s “Large Margin Perceptron” paper
(optional) [ESL] 5.1-5.2, 5.8
Inductive bias and regularization (10/17, 10/19)
slides, 2up
Overfitting in linear regression
(optional) [ESL] 3.4, 12.1-12.3; [PC] 5.11
Dimension reduction (10/24, 10/31, 11/2)
slides, 2up
Notes on SVD
Best fitting line
(optional) Visualization of power method
(optional) Notes eigenvectors/eigenvalues
(optional) [ESL] 14.5.1, 14.5.4-5
Optimization by gradient methods (11/9, 11/14, 11/16)
slides, 2up
Simple implementation of autodiff
Notes on gradient descent (through section 2)
Calibration and bias (11/21)
slides, 2up
COMPAS article
Balanced error rate
Generalization theory (11/28, 11/30)
slides, 2up
(optional) Notes on margins
Neural networks (12/5)
slides, 2up
(optional) LeCun et al’s “Efficient BackProp” paper
(optional) Fleuret’s “Little Book of Deep Learning”

Enrollment

Enrollment and waitlists are managed by the CS department staff. Please do not contact the instructor about enrollment or waitlist issues.

Message from the CS department staff:

While some CS courses are controlled by the instructor, others are overseen by the CS Advising Team to ensure MS students, and upper-level undergraduates (juniors and particularly seniors) who need them for graduation are able to get seats. Given that the waitlists for these classes are so long and that the incoming MS students and UG students are not enrolling until August we need to preserve seats for many different cohorts of students. Now that we have a better understanding of the number of students who will need seats, it’s clear that certain student groups will not be able to secure a seat in these courses at this time. As a result, the decision was made to explicitly deny students who were realistically not going to get seats so that they could use those waitlist slots for other courses at the next registration appointment. You are welcome to re-add yourself to those waitlists when your registration window next opens, but keep in mind that the waitlist “clearing” will continue through to Change of Program. We will do our best to get students into the courses they need.