COMS 6998: Advanced Topics in Spoken Language Processing

Instructors: Julia Hirschberg

Time: Tu 4:10-6:00 (Spring 2020)

Location: TBD

 

Prerequisite: COMS 4705 or another speech or NLP class

Description:  This class will introduce students to spoken language processing:  basic concepts, analysis approaches, and applications.

Required readings:

Jurafsky & Martin 2019 chapters

These and other readings are linked from this syllabus for each class.

Suggested:

Keith Johnson. Acoustic & Auditory Phonetics (3rd edition). Wiley.  2011.

 

Resources:

A list of resources can be found here.

 

Office Hours

Julia Hirschberg: TBD

 

Grade Breakdown

5% attendance and participation

15% weekly posts

15% HW1

25% HW2

25% HW3

 

 

Academic Integrity

The SEAS academic integrity policy is found here.

The CS academic integrity policy is found here.

Syllabus

Note: Schedule and readings may be subject to change

 

Date	Topic	Readings	Assignments
Week 1: 1/21	Introduction to Speech Processing
Week 2: 1/28	From Sounds to Language	Jurafsky & Martin Chapter 7 (sections 1-3)
Week 3: 2/4	Acoustics of Speech	Jurafsky & Martin Chapter 7 (sections 4-7)
Week 4: 2/11	Tools for Speech Analysis	Praat Tutorial(Chapter 11 - scripting - is optional) Download Praat	HW1: Praat Recording and Analysis (assigned)
Week 5: 2/18	Analyzing Speech Prosody	ToBI Conventions Modeling Prosody Prosody and Meaning
Week 6: 2/25	Text-to-Speech Synthesis	Jurafsky & Martin Chapter 8 Merlin Tutorial	HW1 due
Week 7: 3/3	Speech Recognition: Then and Now	Jurafsky & Martin Chapter 9 Deng & Yu Chapter 7
Week 8: 3/10	Spoken Dialogue Systems	Jurafsky & Martin Chapter 24 Jurafsky & Martin Chapter 25	HW2: Dialogue Acts (assigned)
Week 9: 3/17	Spring Break: No Class
Week 10: 3/24	Speech Analysis: Entrainment in Spoken Language	Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions Mark My Words! Linguistic Style Accommodation in Social Media Prosodic entrainment in Mandarin and English: a cross-linguistic comparison
Week 11: 3/31	Speech Analysis: Personality and Mental State	Detecting late-life depression in Alzheimer's disease through analysis of speech and language Vocal-Source Biomarkers for Depresion: A Link to Psychomotor Activity Automatic Recognition of Personality in Conversation	HW2 due
Week 13: 4/7	Speech Analysis: Emotion, Sentiment and Keyword Search	Classifying Subject Ratings of Emotional Speech Using Acoustic Features Using Context to Improve Emotion Detection in Spoken Dialog Systems Adieu features? end-to-end speech emotion recognition using a deep convolutional recurrent network	HW3: Emotional Speech Detection (assigned)
Week 12: 4/14	Speech Analysis: Deception and Trust	Linguistic Cues to Deception and Perceived Deception in Interview Dialogues Lying Words: Predicting Deception from Linguistic Styles Personality Factors in Human Deception Detection: Comparing Human to Machine Performance
Week 14: 4/21	Speech Analysis: Sarcasm and Humor	Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words "Sure, I did the right thing": A system for sarcasm detection in speech "Yeah, right": Sarcasm recognition for spoken dialogue systems	HW3 due
Week 15: 4/28	Speech Analysis: Charisma, Likability and Style	Charisma perception from text and speech "Would You Buy A Car From Me?"-- On the Likability of Telephone Voices Extracting Social Meaning: Identifying Interactional Style in Spoken Conversation