Schedule for COMS 6998-7 Spring 2025

1/27

topic: information sources and measures
assigned reading: Shannon, 1948 (1.2--1.7); Shannon, 1951
optional: Aczel, Forte, Ng, 1974; Brown et al, 1992a
slides

2/3

topic: maximum entropy models
assigned reading: Berger et al, 1996; Dudík et al, 2004
optional: Della Pietra et al, 1997; Csizár and Shields, 2004 (Section 3); Phillips et al, 2004
slides, notes on maxent

2/10

topic: word embedding models
assigned reading: Papadimitriou et al, 2000; *Collins et al, 2001; *Mikolov et al, 2013
strongly recommended but optional: Pereira, 2000 (especially Sections 4 and 6)
optional: Brown et al, 1992b; Stratos et al, 2014; Rudolph et al, 2016; Cotterell et al, 2017
slides

2/17

topic: word embeddings and neural language models
assigned reading: *Gittens et al, 2017 (Sections 1--2); *Bengio et al, 2003; *Collobert et al, 2011
optional: Ando and Zhang, 2005; Gutmann and Hyvärinen, 2012; Arora et al, 2016; Ri et al, 2023
notes on skip-gram model
slides

2/24

topic: neural computation models
assigned reading: *Barron, 1993 (but see below); *Siegelmann and Sontag, 1992; *Hardt and Ma, 2017
optional: Bach, 2024 (Section 9.3; can read this instead of Barron, 1993)
slides

3/3

(no lecture)

3/10

topic: neural computation models
assigned reading: *Demircigil et al, 2017; *Elhage et al, 2021; *Kazemnejad et al, 2023
optional: Krotov and Hopfield, 2016; Alammar, 2018; Sanford et al, 2024b; Guo et al, 2025
slides on associative memories
slides

3/17

(spring break: no class)

3/24

topic: neural computation models
assigned reading: *Liu et al, 2023; *Sanford et al, 2024a; *Wen et al, 2024
optional: Sanford et al, 2023 (Appendix A gives some limitations of feedforward nets and recurrent nets)

3/31

topic: learning theory for neural computation models
assigned reading: *Goldberg and Jerrum, 1995; *Bartlett, 1996
optional: Bartlett and Maass, 2003
slides

4/7

topic: learning theory for neural computation models
assigned reading: *Edelman et al, 2022; Bietti et al, 2023; *Nichani et al, 2024
optional: Li et al, 2024

4/14

topic: preference learning
assigned reading: *Negahban et al, 2016 or *Maystre and Grossglauser, 2015; *Gao et al, 2024
optional: Kakade, 2001; Rafailov et al, 2023
slides

4/21

topic: chain-of-thought
assigned reading: *Rivest and Sloan, 1994; *Merrill and Sabharwal, 2024; *Malach, 2024
slides

4/28

topic: calibration
assigned reading: *Kalai and Vempala, 2024; *Miao and Kearns, 2025
note on Good-Turing

5/5

topic: chain-of-thought, compositional learning
assigned reading: *Joshi et al, 2025; *Valiant, 2000; *Deng et al, 2021