COMS 6998: Topics in Spoken Language Processing

Instructors: Julia Hirschberg and Sarah Ita Levitan

Time: F 10:10-12:00 (Spring 2019)

Location: Mudd 627

 

Prerequisite: COMS 4705 or another speech or NLP class

Description:  This class will introduce students to spoken language processing:  basic concepts, analysis approaches, and applications.

Required readings:

Jurafsky & Martin 2019 chapters

These and other readings are linked from this syllabus for each class.

Suggested:

Keith Johnson. Acoustic & Auditory Phonetics (3rd edition). Wiley.  2011.

 

Resources:

Praat: http://www.fon.hum.uva.nl/praat/

tutorials: http://www.fon.hum.uva.nl/praat/manualsByOthers.html)

Hirschberg, Pragmatics and Prosody Oxford Handbook of Pragmatics, 2017, Chapter 28.

 

Grade Breakdown

Class Participation: TBD

Homeworks: TBD

 

Academic Integrity

The SEAS academic integrity policy is found here.

The CS academic integrity policy is found here.

Syllabus

Note: Schedule and readings are subject to change

 

Date

Topic

Readings

Assignments

Week 1: 1/25

Introduction to Speech Processing

 

Week 2: 2/1

From Sounds to Language

Jurafsky & Martin Chapter 7

Week 3: 2/8

Acoustics of Speech

Jurafsky & Martin Chapter 7

Week 4: 2/15

Tools for Speech Analysis

Praat Tutorial

HW1: Praat Recording and Analysis (assigned)

Week 5: 2/22

Analyzing Speech Prosody

Pragmatics and Intonation (Hirschberg 03)

ToBI Conventions

Modeling Prosody

Prosody and Meaning

HW1 due

Week 6: 3/1

Text-to-Speech Synthesis

Jurafsky & Martin Chapter 8

Merlin Tutorial

HW2: ToBI Labeling (assigned)

Week 7: 3/8

Speech Recognition

Jurafsky & Martin Chapter 9

Deng & Yu Chapter 7

Week 8: 3/15

Spoken Dialogue Systems

 

HW2 due

Week 10: 3/29

Speech Analysis: Entrainment in Spoken Language

Measuring acoustic-prosodic entrainment with respect to multiple levels and dimensions

Mark My Words! Linguistic Style Accommodation in Social Media

HW3: Emotional Speech Detection (assigned)

Week 11: 4/5

Speech Analysis: Emotion, Sentiment and Keyword Search

Classifying Subject Ratings of Emotional Speech Using Acoustic Features

Predicting Student Emotions in Computer-Human Tutoring Dialogues

Using Context to Improve Emotion Detection in Spoken Dialog Systems

Week 12: 4/12

Speech Analysis: Deception and Trust

Combining Acoustic-Prosodic, Lexical, and Phonotactic Features for Automatic Deception Detection

The Detection of Deception: The Effects of First and Second Language on Lie Detection Ability

Personality Factors in Human Deception Detection: Comparing Human to Machine Performance

Lying Words: Predicting Deception from Linguistic Styles

Detecting the Trustworthiness of novel partners in economic exchange

A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction

HW3 due

Week 13: 4/19

Speech Analysis: Sarcasm and Humor

Sarcastic or Not: Word Embeddings to Predict the Literal or Sarcastic Meaning of Words

"Sure, I did the right thing": A system for sarcasm detection in speech

"Yeah, right": Sarcasm recognition for spoken dialogue systems

HW4: Dialogue Acts (assigned)

Week 14: 4/26

Speech Analysis: Charisma, Likability and Style

Charisma perception from text and speech

"Would You Buy A Car From Me?"-- On the Likability of Telephone Voices

Extracting Social Meaning: Identifying Interactional Style in Spoken Conversation

 

Week 15: 5/3

Speech Analysis: Personality and Mental State

Towards Automatically Classifying Depressive Symptoms from Twitter Data for Population Health

Vocal-Source Biomarkers for Depression: A Link to Psychomotor Activity

Computer-based personality judgments are more accurate than those made by humans

Automatic Recognition of Personality in Conversation

Efficacy of a Web-Based, Crowdsourced Peer-To-Peer Cognitive Reappraisal Platform for Depression: Randomized Controlled Trial

HW4 due