Low-level Vision by Consensus in a Spatial Hierarchy of Regions

We introduce a multi-scale framework for low-level vision, where the goal is estimating physical scene values from image data---such as depth from stereo image pairs. The framework uses a dense, overlapping set of image regions at multiple scales and a "local model," such as a slanted-plane model for stereo disparity, that is expected to be valid piecewise across the visual field. Estimation is cast as optimization over a dichotomous mixture of variables, simultaneously determining which regions are inliers with respect to the local model (binary variables) and the correct co-ordinates in the local model space for each inlying region (continuous variables). When the regions are organized into a multi-scale hierarchy, optimization can occur in an efficient and parallel architecture, where distributed computational units iteratively perform calculations and share information through sparse connections between parents and children. The framework performs well on a standard benchmark for binocular stereo, and it produces a distributional scene representation that is appropriate for combining with higher-level reasoning and other low-level cues.Comment: Accepted to CVPR 2015. Project page: http://www.ttic.edu/chakrabarti/consensus

3D model based stereo reconstruction using coupled Markov random fields

Projet SYNTIMA lot of methods have been proposed in the field of stereo-reconstruction. We address here the problem of model-based tridimensional reconstruction from an alreadysegmented and matched stereo pair. This work is a continuation of the work presented, concerning the reconstruction problem. We study here a method based on Markov random fields, which allows the a priori segmentation and matching to be refined during the reconstruction of the 3D surfaces. A new segmentation and matching is then produced which respects the 3D coherence (or equivalently the disparity coherence) of each segmented region-pair. In this first approach, we use simple segmentation energies for each image (without line processes), plus a coupling term between the left and right images, associating planes (as surface primitives) with each region pair. This is the justification for using coupled Markov random fields. We present results on synthetic and real images. These preliminary results allow us to assess the feasability of a hierarchical stereo reconstruction method with no a priori segmentation

Cross-Scale Cost Aggregation for Stereo Matching

Human beings process stereoscopic correspondence across multiple scales. However, this bio-inspiration is ignored by state-of-the-art cost aggregation methods for dense stereo correspondence. In this paper, a generic cross-scale cost aggregation framework is proposed to allow multi-scale interaction in cost aggregation. We firstly reformulate cost aggregation from a unified optimization perspective and show that different cost aggregation methods essentially differ in the choices of similarity kernels. Then, an inter-scale regularizer is introduced into optimization and solving this new optimization problem leads to the proposed framework. Since the regularization term is independent of the similarity kernel, various cost aggregation methods can be integrated into the proposed general framework. We show that the cross-scale framework is important as it effectively and efficiently expands state-of-the-art cost aggregation methods and leads to significant improvements, when evaluated on Middlebury, KITTI and New Tsukuba datasets.Comment: To Appear in 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014 (poster, 29.88%