Contents

• Contents
• List of Figures
• List of Tables
• Introduction
  • Objectives and Features
  • Related Work, Background and Motivation
    • Perception
    • Cognitive Science and Ethology
    • Synthesis: Interactive Environments, Graphics and Robotics
    • Automatic Machine Learning
  • Disclaimer
  • Organization
• Action-Reaction Learning: An Overview of the Paradigm
  • A Typical Scenario
  • Discussion and Properties
    • Perception and Expression
    • Imitation Learning
    • Probabilistic and Bayesian Machine Learning Techniques
    • Cognitive Concepts
• Visual Inputs and Outputs
  • Head and Hand Tracking
    • Glove Tracking
  • Stick Figure Graphical System
  • The Training and Testing Environment
  • Other Sensing and Graphics
• Temporal Modeling
  • Time Series
  • Principal Components Analysis
  • Pre-Processing Implementation and Exponential Memory Decay
  • Probabilistic Time Series Modeling
• Learning: Conditional vs. Joint
  • Machine Learning: Practical Considerations
  • Joint versus Conditional Densities - Pros and Cons
    • Squandered Resources: The Shoe Salesman Problem
  • Conditional Densities: A Bayesian Framework
    • Conditional Density Estimation
• Optimization - General Bound Maximization
  • Bounding for Optimization
    • Quadratic Bounds
  • Maximizing with Bounds: Beyond Gradient Ascent
  • Placing and Tightening the Bound
  • General Bound Maximization and Properties
  • Optimization Examples
    • Fourier Decomposition Optimization
  • Beyond Local Optimization: Annealing the Bounds
  • High Dimensions
• CEM - A Maximum Conditional Likelihood Learner
  • From EM to CEM
    • Discrete Hidden Variables CEM
    • Continuous Hidden Variables CEM
  • CEM for Gaussian Mixture Models
    • Updating the Experts
    • Updating the Gates
    • Bounding Scalars: The Gate Mixing Proportions
    • Bounding Vectors: The Gate Means
    • Bounding Matrices: The Gate Covariances
    • MAP Estimation
  • Implementation and Interpretation
    • Conditional Constraints vs. Joint Constraints
  • Applying the Model
    • Expectation and Arg Max
    • Standardized Database Performance
• An Integrated Learning and Synthesis System
  • System Components and Integration
  • Modularity
  • Human and Human
    • Training Mode
    • Prediction Mode
  • Human and Machine
    • Interaction Mode
    • Perceptual Feedback Mode
  • Machine and Machine
    • Simulation Mode
  • Practical Issues in an Integrated System
• Interaction and Results
  • Application Scenario
  • Evaluation
    • Quantitative Prediction
    • Qualitative Interaction
  • Alternate Applications
    • Alternate Modalities
    • Intra-Modal Learning
    • Non-Human Training Data
  • Limitations
    • Difficulty with Discrete Events
    • Lack of Modularity
    • Constant Length Memory and State
    • Well-Behaved, Smooth Representations are Critical
    • No High Level Goals, Motivations, or Sequences of Actions
• Conclusions and Contributions
  • Current and Future Work
    • Continuous Online Learning
    • Face Modeling for Interaction
  • Conclusions
• Appendix
  • Conditional and Unconditional Bayesian Integration
• Bibliography