Signature of Author
Certified by
Accepted by
Abstract
I propose Action-Reaction Learning as an approach for analyzing and synthesizing human behaviour. This paradigm uncovers causal mappings between past and future events or between an action and its reaction by observing time sequences. I apply this method to analyze human interaction and to subsequently synthesize human behaviour. Using a time series of perceptual measurements, a system automatically uncovers a mapping between past gestures from one human participant (an action) and a subsequent gesture (a reaction) from another participant. A probabilistic model is trained from data of the human interaction using a novel estimation technique, Conditional Expectation Maximization (CEM). The estimation uses general bounding and maximization to find the maximum conditional likelihood solution. The learning system drives a graphical interactive character which probabilistically predicts the most likely response to a user's behaviour and performs it interactively. Thus, after analyzing human interaction in a pair of participants, the system is able to replace one of them and interact with a single remaining user.
The following people served as readers for this thesis:
Reader:
Reader:
Reader:
Acknowledgments
I extend warm thanks to my advisor, Professor Alex Pentland for having given me to opportunity to come to MIT and for being a great source of inspirational ideas, insight and enthusiasm. Thanks, Sandy, for your support, knowledge, wisdom and patience, and for your faith in me.
I thank Professors Aaron Bobick and Bruce Blumberg for having been readers for this thesis. Thanks, Aaron, for your wit, intuition and in-your-face honesty. Thanks, Bruce, for your inspiring ideas and knowledge on animation and behaviour.
I thank Professor Michael Jordan for having read and commented on parts of the thesis. Thanks, Mike, for sharing your passion and knowledge of machine learning and statistics.
I also wish to thank my friends at the Media Laboratory for their support during the thesis. Thanks to Kris Popat who originally motivated me to think about conditional densities and for his inspirational work on decision-tree conditional density estimation. Thanks, Nuria Oliver, for helping edit this thesis and for your cherished support. Thanks, Nitin Sawhney, for working late-night and being there when everybody else was asleep. Thanks, Deb Roy, for showing me the ropes and reminding me to relax. Thanks to Brian Clarkson for hearing me whine about everything. Thanks to Bernt Schiele for reading the thesis and being the best German post-doc in VisMod. Thanks to Sumit Basu for letting me steal his cookies. Thanks, Tom Minka, for reading the thesis and for great conversations about statistics. Thanks, Ken Russell, for help with face modeling, excellent hacking and, of course, blitzing to FedEx. Thanks as well to Chris Wren and Barbara Rosario, office mates who had to deal with me taking so much space and having such a messy desk. Thanks to the VizModGirls: Tanzeem Choudry, Karen Navarro, Professor Rosalind Picard, Flavia Sparacino and Jen Healey. I'm running out of space so I'll speed up... Thanks to Thad Starner, Baback Moghaddam, Crazy Lee Campbell, Martin Zoomy Szummer, Andy Wilson, Claudio Pinhanez, Yuri Ivanov, Raul Fernandez, Chris Bentzel, Ifung Lu, Eric Trimble, Joey Berzowska, Jonathan Klein, Claudia Urrea, Marina Umaschi, Gert-Jan Zwart, Kai Wu, Avinash Sankhla, Dimitri Kountourogiannis, Laxmi Perori, Deepak Jain, Frank Bader and everybody else who I'm forgetting who made this place really fun.
Also, a heart-felt thanks to all my friends from back home in Montreal.
I would most like to thank my family: my father, mother and sister, Carine. They are the best part of my life.