Modeling long-range interactions across the visual field in stereo correspondence

When the eyes are converged, most objects in the visual scene will have a significant vertical disparity as measured at the retina. The pattern of vertical disparity across the retina is largely independent of object depth, depending mainly on the particular eye position adopted. Recently, Phillipson and Read (2010, European Journal of Neuroscience, doi:10.1111/j.1460-9568.2010.07454.x) showed that humans are better at achieving stereo correspondence when the vertical disparity field indicated infinite viewing distance, even when the physical viewing distance was just 30cm. They interpreted this as indicating that disparity encoding is optimized for long viewing distances, and is not updated to reflect changes in eye posture. Their results also indicated a significant effect of the visual periphery. Performance was better when the vertical disparity across the entire visual field was consistent with a given binocular eye position – even when this was not the eye position actually adopted – than when the vertical disparity beyond 20o eccentricity indicated a different eye position than that within 20o eccentricity. This is a surprising result, since (i) the task was to detect a target 8o in diameter, extending from 10o to 18o eccentricity, so information beyond 20o was completely irrelevant to the task, and (ii) many previous results indicate that the visual system detects and uses vertical disparity in local regions, even when the global vertical disparity field is not consistent with any single binocular eye position. Here, I show that this effect can be explained by a template-matching model in which the response of a population of disparity-detectors is compared with stored templates of the response expected to stimuli of known disparity

Parallel stereo vision algorithm

Integrating a stereo-photogrammetric robot head into a real-time system requires software solutions that rapidly resolve the stereo correspondence problem. The stereo-matcher presented in this paper uses therefore code parallelisation and was tested on three different processors with x87 and AVX. The results show that a 5mega pixels colour image can be matched in 5,55 seconds or as monochrome in 3,3 seconds

Phase relationships in stereoscopic computation

We apply the notion that phase differences can be used to interpret disparity between a pair of stereoscopic images. Indeed, phase relationships can also be used to obtain orientation and probabilistic measures from both edges and comers, as well as the directional instantaneous frequency of an image field. The method of phase differences is shown to be equivalent to a Newton-Raphson root finding iteration through the resolutions of band-pass filtering. The method does, however, suffer from stability problems, and in particular stationary phase. The stability problems associated with this technique are implicitly derived from the mechanism used to interpet disparity, which in general requires an assumption of linear phase and the local instantaneous frequency. We present two techniques. Firstly, we use the centre frequency of the applied band-pass filter to interpret disparity. This interpretation, however, suffers heavily from phase error and requires considerable damping prior to convergence. Secondly, we use the derivative of phase to obtain the instantaneous frequency from an image, which is then used to improve the disparity estimate. The second measure is implicitly sensitive to regions that exhibit stationary phase. We prove that stationary phase is a form of aliasing. To maintain stability with this technique, it is essential to smooth the disparity signal at each resolution of filtering. These ideas are extended into 2-D where it is possible to extract both vertical and horizontal disparities. Unfortunately, extension into 2-D also introduces a similar form of the motion aperture problem. The best image regions to disambiguate both horizontal and vertical disparities lie in the presence of comers. Fortunately, we introduce a measure for identifying orthogonal image signals based upon the same filters that we use to interpret disparity. We find that in the presence of dominant edge energy, there is an error in horizontal disparity interpretation that varies as a cosine function. This error can be reduced by iteration or resolving the horizontal component of the disparity signal. These ideas are also applied towards the computation of deformation, which is related to the magnitude and direction of surface slant. This is a natural application to the ideas presented in this thesis

Periodic letter strokes within a word affect fixation disparity during reading

We investigated the way in which binocular coordination in reading is affected by the spatial structure of text. Vergence eye movements were measured (EyeLink II) in 32 observers while they read 120 single German sentences (Potsdam Sentence Corpus) silently for comprehension. The similarity in shape between the neighboring strokes of component letters, as measured by the first peak in the horizontal auto-correlation of the images of the words, was found to be associated with (i) a smaller minimum fixation disparity (i.e. vergence error) during fixation; (ii) a longer time to reach this minimum disparity and (iii) a longer overall fixation duration. The results were obtained only for binocular reading: no effects of auto-correlation could be observed for monocular reading. The findings help to explain the longer reading times reported for words and fonts with high auto-correlation and may also begin to provide a causal link between poor binocular control and reading difficulties. © ARVO

Cortical Dynamics of 3-D Surface Perception: Binocular and Half-Occluded Scenic Images

Previous models of stereopsis have concentrated on the task of binocularly matching left and right eye primitives uniquely. A disparity smoothness constraint is often invoked to limit the number of possible matches. These approaches neglect the fact that surface discontinuities are both abundant in natural everyday scenes, and provide a useful cue for scene segmentation. da Vinci stereopsis refers to the more general problem of dealing with surface discontinuities and their associated unmatched monocular regions within binocular scenes. This study develops a mathematical realization of a neural network theory of biological vision, called FACADE Theory, that shows how early cortical stereopsis processes are related to later cortical processes of 3-D surface representation. The mathematical model demonstrates through computer simulation how the visual cortex may generate 3-D boundary segmentations and use them to control filling-in of 3-D surface properties in response to visual scenes. Model mechanisms correctly match disparate binocular regions while filling-in monocular regions with the correct depth within a binocularly viewed scene. This achievement required introduction of a new multiscale binocular filter for stereo matching which clarifies how cortical complex cells match image contours of like contrast polarity, while pooling signals from opposite contrast polarities. Competitive interactions among filter cells suggest how false binocular matches and unmatched monocular cues, which contain eye-of-origin information, arc automatically handled across multiple spatial scales. This network also helps to explain data concerning context-sensitive binocular matching. Pooling of signals from even-symmetric and odd-symmctric simple cells at complex cells helps to eliminate spurious activity peaks in matchable signals. Later stages of cortical processing by the blob and interblob streams, including refined concepts of cooperative boundary grouping and reciprocal stream interactions between boundary and surface representations, arc modeled to provide a complete simulation of the da Vinci stereopsis percept.Office of Naval Research (N00014-95-I-0409, N00014-85-1-0657, N00014-92-J-4015, N00014-91-J-4100); Airforce Office of Scientific Research (90-0175); National Science Foundation (IRI-90-00530); The James S. McDonnell Foundation (94-40

Spatial constraints of stereopsis in video displays

Recent development in video technology, such as the liquid crystal displays and shutters, have made it feasible to incorporate stereoscopic depth into the 3-D representations on 2-D displays. However, depth has already been vividly portrayed in video displays without stereopsis using the classical artists' depth cues described by Helmholtz (1866) and the dynamic depth cues described in detail by Ittleson (1952). Successful static depth cues include overlap, size, linear perspective, texture gradients, and shading. Effective dynamic cues include looming (Regan and Beverly, 1979) and motion parallax (Rogers and Graham, 1982). Stereoscopic depth is superior to the monocular distance cues under certain circumstances. It is most useful at portraying depth intervals as small as 5 to 10 arc secs. For this reason it is extremely useful in user-video interactions such as telepresence. Objects can be manipulated in 3-D space, for example, while a person who controls the operations views a virtual image of the manipulated object on a remote 2-D video display. Stereopsis also provides structure and form information in camouflaged surfaces such as tree foliage. Motion parallax also reveals form; however, without other monocular cues such as overlap, motion parallax can yield an ambiguous perception. For example, a turning sphere, portrayed as solid by parallax can appear to rotate either leftward or rightward. However, only one direction of rotation is perceived when stereo-depth is included. If the scene is static, then stereopsis is the principal cue for revealing the camouflaged surface structure. Finally, dynamic stereopsis provides information about the direction of motion in depth (Regan and Beverly, 1979). Clearly there are many spatial constraints, including spatial frequency content, retinal eccentricity, exposure duration, target spacing, and disparity gradient, which - when properly adjusted - can greatly enhance stereodepth in video displays

Texture Segregation, Surface Representation, and Figure-ground Separation

A widespread view is that most of texture segregation can be accounted for by differences in the spatial frequency content of texture regions. Evidence from both psychophysical and physiological studies indicate, however, that beyond these early filtering stages,there are stages of 3-D boundary segmentation and surface representation that are used to segregate textures. Chromatic segregation of element-arrangement patterns as studied by Beck and colleagues - cannot be completely explained by the filtering mechanisms previously employed to account for achromatic segregation. An element arrangement pattern is composed of two types of elements that are arranged differently in different image regions (e.g., vertically on top and diagonally on bottom). FACADE theory mechanisms that have previously been used to explain data about 3-D vision and figure-ground separation are here used to simulate chromatic texture segregation data, in eluding data with equiluminant elements on dark or light homogenous backgrounds, or backgrounds composed of vertical and horizontal dark or light stripes, or horizontal notched stripes. These data include the fact that segregation of patterns composed of red and blue squares decreases with inereasing luminance of the interspaces. Asymmetric segregation properties under 3-D viewing conditions with the cquiluminant element;; dose or far arc abo simulated. Two key model properties arc a spatial impenetrability property that inhibits boundary grouping across regions with noncolinear texture elements, and a boundary-surface consistency property that uses feedback between boundary and surface representations to eliminate spurious boundary groupings and separate figures from their backgrounds.Office of Naval Research (N00014-95-1-0409, N00014-95-1-0657, ONR N00014-91-J-4100); CNPq/Brazil (520419/96-0); Air Force Office of Scientific Research (F49620-92-J-0334

Hysteresis in human binocular fusion: temporalward and nasalward ranges

Fender and Julesz [J. Opt. Soc. Am. 57, 819 (1967)] moved pairs of retinally stabilized images across the temporalward visual fields and found significant differences between the disparities that elicited fusion and the disparities at which fusion was lost. They recognized this phenomenon as an example of hysteresis. In the work reported in this paper, binocular retinally stabilized images of vertical dark bars on white backgrounds were moved into horizontal disparity in both the nasalward and the temporalward directions. The limits of Panum's fusional area and the hysteresis demonstrated by these limits were measured for two observers. The following results were obtained: (1) the nasalward limits of Panum's fusional area and the hysteresis demonstrated by the nasalward limits do not differ significantly from the temporalward limits and the hysteresis demonstrated by the temporalward limits; (2) the limits of Panum's fusional area and the hysteresis demonstrated by these limits are not significantly different if one stimulus moves across each retina or if one stimulus is held still on one retina and the other stimulus is moved across the other retina; (3) the use of nonstabilized cross hairs for fixation decreases the hysteresis; and (4) the full hysteresis effect can be elicited with a rate of change of disparity of 2 arcmin/sec

Figure-Ground Separation by Visual Cortex

Office of Naval Research (N00014-95-1-0109, N00014-95-1-0657