|
| We live in a vast sea of ever-changing text with few tools available to help us visualize its meaning. The goal of this research is to bridge the gap between graphics and language by developing new theoretical models and supporting technology to create a system that automatically converts descriptive text into rendered 3D scenes representing the meaning of that text. This builds upon previous work done with Richard Sproat in the WordsEye text-to-scene system (available online at www.wordseye.com). New research areas include: |
|
| Contextual Reasoning: Scenes are often described with oblique contextual references to background settings and ongoing actions. Similarly, properties of a scene's constituent objects can constrain the interpretation of the given text. These and other contextual cues can help resolve ambiguities and build rich, robust models of depicted scenes. |
|
| Lexical Semantic Framework: Our goal is to develop a new representation of lexical meaning which builds on but goes beyond existing frameworks (such as FrameNet's rich representation of verb frames) by incorporating new lexial semantic relations that refer to contextual knowledge. |
|
| Storyboards: Textual descriptions often include shifts in time and location as well as other changes of state. The goal of the system is to recognize such changes and depict them in storyboard fashion. This mimics human cognition, where people may not fill in all details, but mentally skip between salient clusters. |
|
| Knowledge Acquisition: Given the ability to perform basic syntactic and semantic analysis of text and an understanding of the selectional restrictions imposed by context and object properties, the system will acquire scene-related world knowledge from large corpora. |
|
