Schedule for COMS 6998-7 Spring 2025

1/27

topic: information sources and measures
assigned reading: Shannon, 1948 (1.2--1.7); Shannon, 1951
optional: Aczel, Forte, Ng, 1974; Brown et al, 1992a
slides

2/3

topic: maximum entropy models
assigned reading: Berger et al, 1996; Dudík et al, 2004
optional: Della Pietra et al, 1997; Csizár and Shields, 2004 (Section 3); Phillips et al, 2004
slides

2/10

topic: word embedding models
assigned reading: Papadimitriou et al, 2000; *Collins et al, 2001; *Mikolov et al, 2013
strongly recommended but optional: Pereira, 2000 (especially Sections 4 and 6)
optional: Brown et al, 1992b; Stratos et al, 2014; Rudolph et al, 2016; Cotterell et al, 2017
slides

2/17

topic: word embeddings and neural language models
assigned reading: *Gittens et al, 2017 (Sections 1--2); *Bengio et al, 2003; *Collobert et al, 2011
optional: Arora et al, 2016; Ri et al, 2023

2/24

topic: neural computation models
assigned reading: *Barron, 1993; *Siegelmann and Sontag, 1992; *Hardt and Ma, 2017

3/3

talk about projects; other catch-up

3/10

topic: neural computation models
assigned reading: *Demircigil et al, 2017; *Elhage et al, 2021; *Kazemnejad et al, 2023
optional: Krotov and Hopfield, 2016; Alammar, 2018

3/17

(spring break: no class)

3/24

topic: neural computation models
assigned reading: *Liu et al, 2023; *Wen et al, 2024; *Sanford et al, 2024

3/31

topic: learning theory for neural computation models
assigned reading: *Goldberg and Jerrum, 1995; *Bartlett, 1996; (maybe one more paper to come)

4/7

topic: learning theory for neural computation models
assigned reading: *Edelman et al, 2022; *Bietti et al, 2023; *Nichani et al, 2024
optional: Li et al, 2024

4/14

topic: preference learning
assigned reading: *Negahban et al, 2016; *Gao et al, 2024
optional: Kakade, 2001; Rafailov et al, 2023

4/21

topic: chain-of-thought
assigned reading: *Rivest and Sloan, 1994; *Merrill and Sabharwal, 2024; *Malach, 2024

4/28

topic: calibration
assigned reading: *Foster and Kakade, 2006; *Kalai and Vempala, 2024; (maybe one more paper to come)

5/5

topic: compositional learning
assigned reading: *Valiant, 2000; *Deng et al, 2021