Reconstruction and Visualization of Architectural Scenes

Abstract

Can we experience a scene virtually, such as the Colosseum in Rome, without ever having to visit it? Such an experience should replicate the feeling of being physically present, in terms of being able to visualize the scene from different viewpoints as well as quickly assimilating the highlights of the scene. An Internet search can increasingly provide us with a complete photographic record for the scene, but the challenge is in displaying such imagery in a coherent and informative way. In my thesis, I propose an approach for this problem based on 3D reconstruction and path planning. To make this approach feasible, we need to overcome three primary challenges. The first is in scaling the reconstruction algorithms to process millions of 3D points and thousands of images in an efficient manner. We design effective preconditioners to solve the non linear Bundle Adjustment problem efficiently, to obtain significant reductions in execution time. The second challenge involves improving the quality of the 3D reconstructions. Despite decades of research, state-of-the-art stereo algorithms cannot produce quality reconstructions everywhere, due to their dependence on the presence of texture. We complement stereo with monocular cues to overcome this challenge to compute more accurate and complete reconstructions. Finally having computed the reconstructions, a third challenge is in creating compelling visual experiences to aid a user in effectively navigating through the scene. We automatically compute movies, or photo tours, as paths through the reconstruction that are coherent, informative and efficient, for famous sites all over the world.