Changxi Zheng
Physics-Based Sound Rendering for Computer Animation.
Department of Computer Science, Cornell University, August 2012
(Cornell CS Best Dissertation Award) Paper (PDF) Abstract The real world is full of sounds: a babbling brook winding through a tranquil
forest, an agitated shopping cart plugging down a flight of stairs, or a
falling piggybank breaking on the ground. Unfortunately virtual worlds
simulated by current simulation algorithms are still inherently silent. Sounds
are added as afterthoughts, often using "canned sounds" which have little to
do with the animated geometry and physics. While recent decades have seen
dramatic success of 3D computer animation, our brain still expects a full
spectrum of sensations. The lack of realistic sound rendering methods will
continue to cripple our ability to enable highly interactive and realistic
virtual experiences as computers become faster.
This dissertation presents a family of algorithms for procedural sound synthesis
for computer animation. These algorithms are built on physics-based simulation
methods for computer graphics, simulating both the object vibrations for sound
sources and sound propagation in virtual environments. These approaches make
it feasible to automatically generate realistic sounds synchronized with
animated dynamics.
Our first contribution is a physically based algorithm for synthesizing sounds
synchronized with brittle fracture animations. Extending time-varying rigid-body
sound models, this method first resolves near-audio-rate fracture events using a
fast quasistatic elastic stress solver, and then estimates fracture patterns and
resulting fracture impulses using an energy-based model. To make it practical
for a large number of fracture debris, we exploit human perceptual ambiguity when
synthesizing sounds from many objects, and propose to use pre-computed sound
proxies for reduced cost of sound-model generation.
We then introduce a contact sound model for improved sound quality. This method
captures very detailed non-rigid sound phenomena by resolving modal vibrations
in both collision and frictional contact processing stages, thereby producing
contact sounds with much richer audible details such as micro-collisions and
chattering. This algorithm is practical, enabled by a novel asynchronous
integrator with model-level adaptivity built into a frictional contact solver.
Our third contribution focuses on another major type of sound phenomena, fluid
sounds. We propose a practical method for automatic synthesis of bubble-based
fluid sounds from fluid animations. This method first acoustically augments
existing incompressible fluid solvers with particle-based models for bubble
creation, vibration, and advection. To model sound propagation in both fluid and air
domain, we weight each single-bubble sound by its bubble-to-ear acoustic
transfer function value, which is modeled as a discrete Green's function of the
Helmholtz equation. A fast dual-domain multipole boundary-integral solver is
introduced for hundreds of thousands of Helmholtz solves in a typical babbling
fluid simulation.
Finally, we switch gear and present a fast self-collision detection method for
deforming triangle meshes. This method can accelerate deformable simulations and
lead to faster sound synthesis of deformable phenomena. Inspired by a
simple idea that a mesh cannot self collide unless it deforms enough, this
method supports arbitrary mesh deformations while still being fast. Given a
bounding volume hierarchy (BVH) for a triangle mesh, we operate on bounding-volume-related submeshes, and
precompute Energy-based Self-Collision Culling (ESCC) certificates,
which indicate the amount of deformation energy required for the submesh to self collide.
After updating energy values at runtime, many bounding-volume self-collision
queries can be culled using the ESCC certificates. We propose an affine-frame
Laplacian-based energy definition which sports a highly optimized certificate
preprocess and fast runtime energy evaluation.