It's not necessary to have one brewski too many to
glimpse menus floating in front of restaurants a block
away, direction signs where there aren't any, or
buildings that are long gone. All you need is augmented
reality.
For years now, researchers have been working to
overlay computer-generated images onto the real world to
inform, assist, reconstruct. You may not realize it, but
you're already familiar with a form of augmented
reality: the yellow first-down line on football
telecasts and those ads behind home plate seen by TV
viewers but not by fans at the baseball stadium.
The creators of these effects often refer to them as
"virtual" enhancements, in keeping with the terminology
for virtual reality--a technology first developed in the
1980s. Virtual reality aimed to substitute
computer-generated visuals for the real world; augmented
reality (as its name implies) merely adds graphics to
existing surroundings.
While computer scientists are busy perfecting
augmented reality in labs as far afield as North
Carolina and Washington state, it's only fitting that
our own hallucinatory hamlet is an important center for
this new way of seeing.
The Sporting Views
During football season, the CBS Broadcast
Center on W. 57th St. has a room entirely devoted to
equipment for adding the first-down line to as many as
seven regional games being aired simultaneously. At
other times, this hardware is dispersed elsewhere to
overlay baseball broadcasts with backstop ads or golf
coverage with computer-created paths of shots.
PVI Virtual Media Services in Lawrenceville, N.J.,
provides augmented reality technology to CBS. PVI
competitor Sportvision, pioneer of the "1st-and-10" line
and other enhancements, adds these to broadcasts for
ABC, Fox, and ESPN.
In augmented reality, point of view is all-important.
PVI uses sensor-independent vision processing,
essentially computer recognition of what's on the
football field. In the control room, a technician loads
an unenhanced video tape of a recent game and cues a
6-foot-high rack of computer hardware named LVIS (for
Live Video Insertion System).
"It's looking for the yard lines and working out
which lines it's looking at," explains Lionel White, PVI
vice president for operations. LVIS does this not by
reading the numbers on the field, but mathematically,
according to the lines' perspective. The technician keys
in the number of yards from where the ball is placed to
a first down and the familiar yellow stripe appears on
screen as if it were painted across the width of the
field.
High-Tech Ghosts
What if, instead of needing bulky hardware
and TV monitors for millions of viewers, augmented
reality could be implemented on portable computers and
lightweight goggles for personal use?
That's what Prof. Steven Feiner of Columbia
University has been working on for more than a decade.
He and his students have developed an evolving platform
for testing applications called Mobile Augmented Reality
Systems (MARS) - with fund-ing from the Office of Naval
Research.
The public will have a rare opportunity to experience
MARS technology on May 27-28, during Fleet Week, onboard
the Afloat Lab ship at Pier 86 by the USS Intrepid on
the Hudson River. In a simple demonstration, visitors
donning display goggles will be able to look around the
cabin and see the computer-generated equivalent of
Post-It Notes floating beside features of interest.
Imagine repair procedures overlayed on complicated
machinery or markers of hazards below the water or over
the horizon, and the reasons for the Navy's sponsorship
become clear.
Feiner and his team have also created intriguing
civilian applications. A computer-aided tour of the
Columbia University campus presents informational signs
and on-the-spot video documentaries through the goggles,
opening the way for augmented reality for tourism. A
restaurant guide offering menus and reviews hovering by
restaurants on Broadway is the precursor of a futuristic
Zagat's. And a recently completed visualization of a dig
in Sicily allows archaeology students to interact
realistically with images of artifacts as they were
discovered thousands of miles away.
Establishing a user's point of view currently
requires cumbersome equipment, arrays of
sensors indoors or a GPS and motion-sensing system
outdoors, plus goggles. But Feiner envisions a time when
an ordinary-looking pair of glasseswith a built-in
display, and a compact system that recognizes
faces,could make augmented reality a social necessity.
"It could show you the name of a person you run into,
remind you of personal information, and conversations
you've had."
Next on Feiner's agenda is computer modeling of a
nearby landmark building as it appears today and as it
did at different stages of construction and in plans
that were never executed.
"In augmented reality," he says, "you could turn back
the clock to see visual reconstructions of how things
looked at that location at various times."
For instance, the same technology could be used
someday to recreate an onsite depiction of the World
Trade Center in all its soaring, tragic majesty.
Originally published on May 16,
2004
