This project is geared towards the development of catadioptric (lens + mirror) imaging systems with unusually large fields of view. One important design goal in catadioptric imaging is choosing the shapes of the mirrors in a way that ensures that the complete imaging system has a single effective viewpoint. The reason a single viewpoint is desirable is that it is a requirement for the generation of pure perspective images from the sensed images. In the first part of the project, we have derived the complete class of single-lens single-mirror catadioptric sensors that have a single viewpoint. We have derived expressions for the spatial resolution of a catadioptric camera in terms of the resolution of the cameras used to construct it. In addition, we have analyzed the defocus blur caused by the use of a curved mirror in a catadioptric sensor.

In the second stage of the project, a framework was developed for the design and analysis of single-viewpoint catadioptric cameras that use two or more mirrors. The use of multiple mirrors permits folding of the optics which leads to more compact camera designs than ones that use a single mirror. A dictionary of camera designs that use two conic mirrors has been created. We have shown that any folded system that uses conic mirrors has a geometrically equivalent system that uses a single conic mirror. This result makes it easy to determine the scene-to-image mapping of a conic folded system.

Over the course of this project, we have built several prototype cameras, each one designed to meet the needs of a specific application. Some of these prototypes have been turned into commercial products by RemoteReality Inc. Some of the images and videos shown on the page are courtesy of RemoteReality, Inc.

Publications

"Folded Catadioptric Cameras,"
S.K. Nayar and V.N. Peri,
Panoramic Vision, pp. 103-119, R., Springer-Verlag, Apr. 2001.
[PDF] [bib] [©]

"Single Viewpoint Catadioptric Cameras,"
S. Baker and S.K. Nayar,
Panoramic Vision, pp. 39-71, R., Springer-Verlag, Apr. 2001.
[PDF] [bib] [©]

"A Theory of Single-Viewpoint Catadioptric Image Formation,"
S. Baker and S.K. Nayar,
International Journal on Computer Vision,
Vol. 35, No. 2, pp. 175-196, Nov. 1999.
[PDF] [bib] [©]

"Folded Catadioptric Cameras,"
S.K. Nayar and V. Peri,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
Vol. 2, pp. 217-223, Jun. 1999.
[PDF] [bib] [©]

"A Theory of Catadioptric Image Formation,"
S. Baker and S.K. Nayar,
IEEE International Conference on Computer Vision (ICCV),
pp. 35-42, Jan. 1998.
[PDF] [bib] [©]

"Catadioptric Omnidirectional Camera,"
S.K. Nayar,
IEEE Conference on Computer Vision and Pattern Recognition (CVPR),
pp. 482-488, Jun. 1997.
[PDF] [bib] [©]

"Generation of Perspective and Panoramic Video from Omnidirectional Video,"
V.N. Peri and S.K. Nayar,
DARPA Image Understanding Workshop (IUW),
pp. 243-246, May. 1997.
[PDF] [bib] [©]

Gallery

Images

		OneShot 360: This camera was developed by RemoteReality based on the technology developed at the CAVE laboratory. It uses a parabolic mirror and a telecentric lens.
		Captured Image: This is a raw 360 degree image captured using the OneShot camera. As one can see, the scene is highly dynamic and it is not possible to capture a 360 degree image of such a scene using the traditional approach of mosaicing a series of images taken at different instances of time.
		Computed Panorama: The captured donut-shaped image is mapped to a spherical panorama. Once this is done existing software tools like QTVR can be used to generate perspective views and navigate around the complete field of view.
		Alaska: Another example of a captured image and the corresponding spherical panorama.

Videos

		360 degree Video Conferencing: This video shows the advantage of using a 360 degree camera for video conferencing. The camera is placed at the center of a round table around which the participants sit. A single video stream captures all the participants and is transmitted to a remote site where a PC can be used to create perspective views of each of the participants in real time. (With audio)
		Driver Assistance: In this video a 180 degree catadioptric camera is attached to the back of a truck. The panoramic video produced by the camera is presented to the driver who gets a full view of what goes on behind the truck.
		Scuba Diving in 360 degrees: This video clip was captured by Terry Boult at the University of Colorado, Colorado Springs, using a water-proof enclosure for the 360 degree camera.
		Immersive Video: In this video, a user wears a head-mounted display to experience the 360 degree underwater video shown in the previous clip. As the user looks around, the appropriate perspective views are computed and presented to the user via the head-mounted display.

Related Projects

Applications of 360 Degree Cameras

The World In An Eye

Spherical Mosaics: Regular and Stereoscopic

Catadioptric Stereo: Planar and Curved Mirrors

Wide Angle Lenses and Polycameras

Non-Single Viewpoint Imaging: Raxels and Caustics