Quick and energy-efficient Bayesian computing of binocular disparity using stochastic digital signals

Reconstruction of the tridimensional geometry of a visual scene using the binocular disparity information is an important issue in computer vision and mobile robotics, which can be formulated as a Bayesian inference problem. However, computation of the full disparity distribution with an advanced Bayesian model is usually an intractable problem, and proves computationally challenging even with a simple model. In this paper, we show how probabilistic hardware using distributed memory and alternate representation of data as stochastic bitstreams can solve that problem with high performance and energy efficiency. We put forward a way to express discrete probability distributions using stochastic data representations and perform Bayesian fusion using those representations, and show how that approach can be applied to diparity computation. We evaluate the system using a simulated stochastic implementation and discuss possible hardware implementations of such architectures and their potential for sensorimotor processing and robotics.Comment: Preprint of article submitted for publication in International Journal of Approximate Reasoning and accepted pending minor revision

Deformable kernels for early vision

Early vision algorithms often have a first stage of linear-filtering that `extracts' from the image information at multiple scales of resolution and multiple orientations. A common difficulty in the design and implementation of such schemes is that one feels compelled to discretize coarsely the space of scales and orientations in order to reduce computation and storage costs. A technique is presented that allows: 1) computing the best approximation of a given family using linear combinations of a small number of `basis' functions; and 2) describing all finite-dimensional families, i.e., the families of filters for which a finite dimensional representation is possible with no error. The technique is based on singular value decomposition and may be applied to generating filters in arbitrary dimensions and subject to arbitrary deformations. The relevant functional analysis results are reviewed and precise conditions for the decomposition to be feasible are stated. Experimental results are presented that demonstrate the applicability of the technique to generating multiorientation multi-scale 2D edge-detection kernels. The implementation issues are also discussed

Analog VLSI implementation for stereo correspondence between 2-D images

Many robotics and navigation systems utilizing stereopsis to determine depth have rigid size and power constraints and require direct physical implementation of the stereo algorithm. The main challenges lie in managing the communication between image sensor and image processor arrays, and in parallelizing the computation to determine stereo correspondence between image pixels in real-time. This paper describes the first comprehensive system level demonstration of a dedicated low-power analog VLSI (very large scale integration) architecture for stereo correspondence suitable for real-time implementation. The inputs to the implemented chip are the ordered pixels from a stereo image pair, and the output is a two-dimensional disparity map. The approach combines biologically inspired silicon modeling with the necessary interfacing options for a complete practical solution that can be built with currently available technology in a compact package. Furthermore, the strategy employed considers multiple factors that may degrade performance, including the spatial correlations in images and the inherent accuracy limitations of analog hardware, and augments the design with countermeasures

SIFT Flow: Dense Correspondence across Scenes and its Applications

While image alignment has been studied in different areas of computer vision for decades, aligning images depicting different scenes remains a challenging problem. Analogous to optical flow where an image is aligned to its temporally adjacent frame, we propose SIFT flow, a method to align an image to its nearest neighbors in a large image corpus containing a variety of scenes. The SIFT flow algorithm consists of matching densely sampled, pixel-wise SIFT features between two images, while preserving spatial discontinuities. The SIFT features allow robust matching across different scene/object appearances, whereas the discontinuity-preserving spatial model allows matching of objects located at different parts of the scene. Experiments show that the proposed approach robustly aligns complex scene pairs containing significant spatial differences. Based on SIFT flow, we propose an alignment-based large database framework for image analysis and synthesis, where image information is transferred from the nearest neighbors to a query image according to the dense scene correspondence. This framework is demonstrated through concrete applications, such as motion field prediction from a single image, motion synthesis via object transfer, satellite image registration and face recognition

A localized correlation function for stereoscopic image matching

We propose and study a localized correlation function for stereoscopic matching . The latter is based on the wavelet decomposition of the input images . Contrarily to « coarse-to-fine » algorithms, this one simultaneously processes information at the different scales . The localized correlation function is defined by locally integrating with respect to the scale variable . We show that it is equivalent to the definition of a correlation kernel, which is extremely precise in terms of position and disparity . The definition can then be modified in order to account for the local frequency content of the images . Then, we suggest pre-processings of the images : we argue in favour of the use of multiresolution contrast techniques, associated to a quadratic normalization .Nous proposons et étudions une fonction de corrélation localisée permettant la mise en correspondance d'images stéréoscopiques. Celle-ci est fondée sur la décomposition en échelles des images d'entrée. A l'inverse des algorithmes de type « coarse-to-fine », celui-ci traite simultanément les informations des différentes bandes de fréquence. Pour cela, nous définissons en chaque point des deux images une fonction de corrélation localisée par intégration sur le paramètre d'échelle, dont nous montrons qu'elle est équivalente à la définition d'un noyau de corrélation extrêmement fin dans les paramètres de position et de disparité. La définition peut ensuite être modifiée pour prendre en compte la composition fréquentielle locale des images d'entrées. Enfin, nous nous intéressons au problème de la normalisation préalable des images à apparier et justifions le choix du contraste multirésolution associé à une normalisation quadratique

Shape from Stereo Using Fine Correlation: Method and Error Analysis

We are considering the problem of recovering the three-dimensional geometry of a scene from binocular stereo disparity. Once a dense disparity map has been computed from a stereo pair of images, one often needs to calculate some local differential properties of the corresponding 3-D surface such as orientation or curvatures. The usual approach is to build a 3-D reconstruction of the surface(s) from which all shape properties will then be derived without ever going back to the original images. In this paper, we depart from this paradigm and propose to use the images directly to compute the shape properties. We thus propose a new method, called \emph{fine correlation}, extending the classical correlation method to estimate accurately both the disparity and its derivatives directly from the image data. We then relate those derivatives to differential properties of the surface such as orientation and curvatures. We present the results of the reconstruction and of the estimation of the surface orientation and curvatures on some stereo pairs of real images. The performance and accuracy of both classical and fine correlation are analysed using synthetic images. By modeling the correlation error distribution as a mixture of Gaussian, we can compute the intrinsic accuracy of each method and the proportion of false matches, depending on the local disparity slope. The results show that the accuracy is almost constant with respect to slope using fine correlation, whereas it increases dramatically using classical correlation

Stereoscopic Surface Interpolation from Illusory Contours

Stereoscopic Kanizsa figures are an example of stereoscopic interpolation of an illusory surface. In such stimuli, luminance-defined disparity signals exist only along the edges of inducing elements, but observers reliably perceive a coherent surface that extends across the central region in depth. The aim of this series of experiments was to understand the nature of the disparity signal that underlies the perception of illusory stereoscopic surfaces. I systematically assessed the accuracy and precision of suprathreshold depth percepts using a collection of Kanizsa figures with a wide range of 2D and 3D properties. For comparison, I assessed similar perceptually equated figures with luminance-defined surfaces, with and without inducing elements. A cue combination analysis revealed that observers rely on ordinal depth cues in conjunction with stereopsis when making depth judgements. Thus, 2D properties (e.g. occlusion features and luminance relationships) contribute rich information about 3D surface structure by influencing perceived depth from binocular disparity

A Computational Framework for Determining Stereo Correspondence from a Set of Linear Spatial Filters

We present a computational framework for stereopsis based on the outputs of linear spatial filters tuned to a range of orientations and scales. This approach goes beyond edge-based and area-based approaches by using a richer image description and incorporating several stereo cues that have previously been neglected in the computer vision literature