I am curently a research scientist at Interactions Corporation (who bought Speech&Language Research from AT&T in December 2014). I am also a Visiting Researcher at the Spoken Language Processing Group at Columbia University, working with Professor Julia Hirschberg.

** I am teaching a seminar on Spoken Dialogue Systems in the Spring 2015 **

Research Interests

My research interests lie in the field of dialogue, both human-human and human-computer. I am interested in studying how spoken language can enhance human-computer communication and open doors to new types of intelligent systems that seamlessly integrate in our daily environment. I am also interested in applications of language technology for education.

While people communicate effortlessly with each other, automatic human-computer dialogue is a challenge currently addressed by many researchers. I am interested in analysing human-human dialogue and applying some of its models to human-computer spoken interaction.

Current Projects

Semantic Parsing aims to extract meaning from sentences by identifying targets, arguments, and their relations in a sentence. Semantic parsing is more general than syntax and more specific than sentence intent. A syntactic parse describes relations between all words in a sentence. Intent describes a sentence with a single tag. Semantic parsing, on the other hand, identifies main points of the sentence in a structural form. I applied Tag&Parse approach to semantic parsing for SemEval Task6 competition - Semantic parsin of robot commands. Tag&Parse approach first applies semantic tags to identify arguments, which can be adapted to a domain and then determines structural relations between these arguments. The advantage of the approach is that it can be resource-light and highly accurate for a chosen domain.

My current work focuses on handling misrecognitions in spoken dialogue. Handling speech recognition errors is an important challenge for automatic spoken dialogue systems. Current state-of-the art dialogue systems ask generic clarification questions, such as "Please repeat" or "Please rephrase" when they fail to recognize a user utterance. Clarification questions are also used by speakers in human dialogue. However, in human dialogue, speakers are more likely to ask targeted clarification questions rather than explicitly signal their non-understanding. For example, if an utterance is "Please find a XXX" (XXX refers to a misheard word), an example of a targeted clarification question is "Find what?" or "What should I find?". In this project, we investigate how human speakers handle situations with missing information, how they construct clarification questions and how they answer different types of clarification questions. We use this information to construct targeted clarification questions automatically from an user utterance that contains a speech recognition error.

To be able to ask a targeted clarification question, a system has to detect which part of an utterance is misrecognized. In our work, we address the problem of localized error detection in Automatic Speech Recognition (ASR) output. Localized error detection seeks to identify which particular words in a user's utterance have been misrecognized. We use prosodic, syntactic, semantic, and speech recognizer's confidence features from an utterance and apply machine learning methods to determine which words in an utterance are misrecognized.

I am also interested in automatic dialogue generation. In my previous project (CODA), in collaboration with Paul Piwek at the Open University in the UK, I developed a system for automatic conversion of dialogues from text. Generated dialogues present the same information as original monologues. We constructed a parallel corpus of monologues and dialologues and automatically derived rules for translating monologue text into dialogue. In collaboration with Pascal Kuyten and Helmut Prendinger at NII laboratory in Japan, we work on visualizing these dialogues using virtual characters and animation. The product of this research may be applied in an education tool or gaming environment. I am currently investigating the impact of discourse annotation on quality of automatically generated dialogue.

Students

I supervise graduate and undergaduage students in ongoing research projects:

  • Mei-Vern Then (Undergraduate, Computer Science, Columbia University)
  • Alex Liu (Undergraduate, Computer Science, Columbia University)

Previously supervised students:

  • Eli Pincus (Masters, Applied Mathematics, Columbia University)
  • Rashmi Raman (Masters, Journalism and Computer Science joint program, Columbia University)
  • Ananta Padney (Undergraduate, Computer Science, Barnard Colledge)
  • Jingbo Yaung (MS student, Engineering and Applied Science, Columbia University)
  • Philipp Salletmayer(MS student, Computer Science, Graz University)

Previous Research

In my previous research I have built automatic spoken dialogue systems, studied the effect of the system on the user's lexical and syntactic choices. I have analysed adaptation (speakers changing their behaviour) in human-human dialogues. I have studied how dialogue context can help spoken dialogue systems in speech recognitions.


Contact info:
AT&T Labs Research
33 Thomas St.
New York, NY 10027
Email: svetlana.stoyanchev @ gmail.com