6.864: Natural Language Processing

[Main] | [General Information] | [Problem Sets]

Instructor: Michael Collins
Time & Location: Tues & Thurs 1-2.30, 32-144
Office Hours: By appointment

TA: Igor Malioutov, igorm AT csail.mit.edu

Announcements:

[12/10] Please fill out the class evaluation at this link. The deadline is 11.59pm, December 16th. Thanks! Update: my email from HKN says that evaluations are open until the 16th, but the notices they've posted in Stata say the 14th, it's probably safest to do them by the 14th.
[11/30] The deadline for the final project report has been extended to Friday, 7th December

[11/15] Problem set 4 has been released

[10/30] A histogram of the mid-term scores. Any score of 85/120 or above is an A; 65/120 or above is a B; 50/120 or above is a C.
[10/29] The final exam is scheduled for Wednesday, 19th December, 9am-12 noon, in room 56-154.

[10/29] Details about the Projects have been released. Please read this carefully. Project proposals are due this Friday, i.e., November 2nd. Note also that class on November 8th is cancelled. Instead, I'll be meeting with everybody to discuss their projects.

[10/25] Problem set 3 has been released

[10/16] The mid-term will be on Tuesday October 23rd, 7.30-9.30pm, in room 32-141. The policy for the midterm is to allow you to bring lecture notes from class, as well as any additional notes you've written yourself to supplement this material. We will not allow you to bring other materials to the exam (for example, you can't bring an entire book such as Manning and Schuetze, nor can you bring individual chapters from a book). The mid-term could potentially include any material from the lectures up to and including 10/16 except: 1) the EM lectures (on 10/2 and 10/4) will be excluded (they will be tested on the final). 2) For the lecture on 10/16, only material up to slide 23 will be included. Here are previous mid-terms for Fall 2005 and Fall 2006. Some questions on these mid-terms are on material we have not covered. Specifically, ignore questions 2, 3, and 8 from the 2005 mid-term, and ignore questions 2, 3, 8, 9, and 10 from the 2006 mid-term. The solution for the 2005 mid-term is here. Note again, class on October 23rd is cancelled.
[10/4] Problem set 2 has been released

[10/4] The mid-term will be on the evening of October 23rd. Details (time and location) to be announced soon. Class on October 23rd will be cancelled.

[9/25] Update: We have made a revision to the late policy for problem sets: we will give students 5 "free" days that can be used as they wish across the 4 problem sets. Specifically, we will not penalise the first 5 late days that a student incurs on problem sets. After that, the penalties posted on the problem sets will apply (e.g., 5 points per day late on the first problem set). The final (0 point) deadline will still apply; for example for pset 1 any solutions handed in after October 1st will get 0 points.
[9/20] Please sign up for the class mailing list by following this link.
[9/14] Problem set 1 has been released

[9/6] A link to the 2006 webpage for 6.864

Date	Topic	References
9/6	Part 1: Introduction and Overview Part 2: Language Modeling	Here are some references on language modeling.
9/11	Parsing and Syntax 1
9/18	Parsing and Syntax 2	Chapter 14 (draft) of Jurafsky and Martin is available here. It covers a lot of the material from Parsing lectures 1, 2, and 3. As additional reading, the Charniak (1997) paper is here. A journal paper describing the Collins (1997) model is here.
9/20	Parsing and Syntax 3
9/25	Log-linear models
9/27	Tagging	Background Reading: Daniel M. Bikel, Richard Schwartz and Ralph M. Weischedel. 1999. An Algorithm that Learns What's in a Name. In Machine Learning, Special Issue on Natural Language Learning. Background Reading: Andrew McCallum, Dayne Freitag and Fernando Pereira. Maximum Entropy Markov Models for Information Extraction and Segmentation. In proceedings of ICML 2000. Background Reading: Adwait Ratnaparkhi. A Linear Observed Time Statistical Parser Based on Maximum Entropy Models In proceedings of EMNLP 1997.
10/2	The EM algorithm, part 1
10/4	The EM algorithm, part 2	Required reading for today's lecture is here. You can read this either before or after the class. Here's a brief note clarifying some of the identities in section 5.2.
10/11	Machine Translation, Part 1	Jurafsky and Martin, Chapter 25 sections 25.1-25.3, and 25.9, cover much of the material covered in lecture. If you're interested in reading more about the Bleu evaluation measure, the original paper is here. Another interesting paper on Bleu scores is here.
10/16	Machine Translation, Part 2	Jurafsky and Martin, Chapter 25 sections 25.5.1 and 25.6.1 cover IBM Model 1. The original IBM paper is here. This is definitely not required reading, but you might find it interesting. Here is a very cool article on statistical MT, by Kevin Knight -- again, not required reading but you might find it helpful.
10/18	Machine Translation, Part 3	Here are the slides from Philipp Koehn's tutorial. We covered slides 103-108 (extracting phrases) and slides 29-57 (decoding) in lecture. Jurafsky and Martin, Chapter 25 section 25.4 covers phrase-based models. Section 25.8 covers decoding phrase-based models. Figure 25.30, which shows the decoding algorithm, is particularly important.
10/25	Machine Translation, Part 4 Slides on reordering approaches
10/30	Machine Translation, Part 5	Background reading: The paper by David Chiang is here. Sections 1-3 are most relevant for what we covered in class. Additional reading: This paper might be of interest if you're interested in how s-CFG approaches can make use of treebank parses.
11/1	Global Linear Models, Part 1
11/6	Global Linear Models, Part 2	(Note: we didn't cover slides 30-36.) This paper describes the perceptron algorithm for tagging.
11/13	Guest lecture by Jim Glass: Speech Recognition
11/15	Similarity Measures and Clustering
11/20	Computational models of discourse	Note: slides 1-42 were covered in the lecture.
11/27	Word-sense disambiguation	The paper by David Yarowsky on semi-supervised methods is here. The paper on named-entity classification is here.
11/29	Global Linear Models, Part 3: Dependency Parsing	Chapter 3 of Ryan Mcdonald's thesis has an explanation of the dynamic programming algorithm for dependency parsing.
12/6, 12/11	Learning of CCGs