Minimal Aspect Distortion (MAD) Mosaicing of Long Scenes

Minimal Aspect Distortion (MAD) Mosaicing of Long Scenes


Alex Rav-AchaGiora Engel Shmuel Peleg 

Abstract


Long scenes can be imaged by far away cameras, or by mosaicing multiple images from closer cameras scanning the scene. We address the case of a video camera scanning a scene while moving in a long path, e.g. scanning a city street from a driving car, or scanning a terrain from a low flying aircraft.

A new minimal aspect distortion (MAD) mosaicing of the long scene is presented, which uses depth to minimize the geometrical distortions and to improve stitching of long panoramic images. We also present a robust "direct" method to provide both camera motion and dense depth. A limited motion model of a scanning camera is used to increase robustness of motion computation. Iterative graph-cuts, with planar labels and a flexible similarity measure, is used for depth computation.

Interactive visualization using X-Slits is demonstrated given the computed motion and depth. In this case the depth information is used for better stitching of the mosaic.

The presented approach is very robust, and works on sequences having thousands of frames even when using a hand-held camera. Examples are shown on a few challenging sequences.

Technical Report describing this work (pdf)


Demos
(Click on pictures to enlarge or to view the video)


A Street in Jerusalem

 
Input Video

 
MAD Mosaic and Depth (High Resolution)

 
X-Slits Mosaic and Depth Map

 
MAD Mosaic and Depth Map

 
Synthesized Video: X-Slit (1)

 
Synthesized Video: X-Slit (2)

 

Boat Ride in Germany

 


Input Video

 
X-Slits Mosaic and Depth Map

 
Synthesized Video: Virtual Tour using X-slits

 
Synthesized Video: Stereo Panorama

 

Derailed Shinkansen Train

 


Input Video

 
X-Slits Mosaic and Depth Map 

 
Synthesized Video: Stereo Panorama

 
X-Slits Mosaic of Other Side (High Resolution)