Using 3D Information to Avoid Foreground Objects in Multi-Viewpoint Panoramas

Abstract

In this master thesis, we introduce a proof-of-concept method to aid in foreground object removal for multi-viewpoint panoramas using both sparse and dense stereo information. The approach we extend from is able to generate seamless panoramas with these properties, but this often requires user intervention in an interactive refinement step. We try to reduce the amount of work the user has to perform by identifying and avoiding foreground objects automatically. We aim to assemble a seamless mosaic of a user-selected plane, while eliminating obstructing foreground objects as much as possible. To achieve our goal, we use Markov Random Field optimization to minimize a cost function, where one of the data terms is a new depth-based function which favors imagery close to or behind the picture surface. Depth information is inferred both from the sparse 3D point cloud generated in the initial reconstruction stage, and from stereo disparity maps computed by applying a dense stereo algorithm to the remapped source images. We demonstrate that our approach works for real images, and improves over the results of the method our research was extended from.