Using The Symmetry Of False Matches To Solve The Correspondence Problem

Veridical stereoscopic depth depends on matching corresponding image points. This requires solving the stereo correspondence problem: how are true matches distinguished from false ones? Conventional algorithms select true matches on the basis of feature detection [what do you mean by ‘feature detection’ here? Is there some more specific term?] and adherence to natural statistics. They reject false matches as noise. We propose here an alternative that uses the signals present in false matches to delineate the true solution. When visualized in a Keplerian array, binocular matches are symmetrically reflected about an axis that is a potential solution. Properties such as extent and curvature of the solution are encoded the transformation that describes how one-half of the matches reflects onto the other. To implement this strategy, left and right images were convolved with Gaussian kernels of various standard deviations (spatial frequencies). Keplerian arrays comparing filter responses across left and right spatial-frequency combinations were then constructed. Responses that are minimally different across the eyes give rise to regions of high symmetry; position within the Keplerian array indicates the location of a solution in space. Solutions that possess natural surface regularities consistently showed minimal differences for one left : right spatial frequency ratio, which is correlated with local surface slant. As a result, combining responses within particular ratio families can distinguish true matches from false ones. True matches tend to be long and smoothly contoured, and symmetry would be preserved across all members of a ratio family from low to high spatial-frequency combinations. This approach is efficient; preprocessing is minimal since no feature extraction is involved. It can be implemented in machine vision to solve the correspondence problem for depth sensing algorithms. It is robust when tested against perfectly camouflaged surfaces in random dot stereograms and consistent with physiological data showing that false match signals are propagated to higher cortical areas along the dorsal pathway

Attentional selection in judgments of stereo depth

Stereoscopic depth is most useful when it comes from relative rather than absolute disparities. However, the depth perceived from relative disparities can vary with stimulus parameters that have no connection with depth or are irrelevant to the task. We investigated observers’ ability to judge the stereo depth of task-relevant stimuli while ignoring irrelevant stimuli. The calculation of depth from disparity differs for 1-D and 2-D stimuli and we investigated the role this difference plays in observers’ ability to selectively process relevant information. We show that the presence of irrelevant disparities affects perceived depth differently depending on stimulus dimensionality. Observers could not ignore disparities of irrelevant stimuli when they judged the relative depth between a 1-D stimulus (a grating) and a 2-D stimulus (a plaid). Yet these irrelevant disparities did not affect judgments of the relative depth between 2-D stimuli. Two processes contributing to stereo depth were identified, only one of which computes depth from a horizontal disparity metric and permits attentional selection. The other uses all stimuli, relevant and irrelevant, to calculate an effective disparity direction for comparing disparity magnitudes. These processes produce inseparable effects in most data sets. Using multiple disparity directions and comparing 1-D and 2-D stimuli can distinguish them

False-Match Symmetry: Data Files and Simulation Code

This record provides data from random-dot stereograms to use in solving the binocular correspondence problem through false-match symmetry. It also provides an implementation of the algorithm used in the article ‘Solving the stereo correspondence problem with false matches’ [Ng CJ, Farell B (2019) Solving the stereo correspondence problem with false matches. PLoSONE 14(7): e0219052. https://doi.org/10.1371/journal.pone.0219052]. The record consists of two parts, Data and Algorithm Demo. The Data component consists of pre-computed Keplerian arrays of all possible matches between filtered random-dot image pairs containing stereoscopically defined surfaces. The Algorithm Demo allows data files to be computed afresh from supplied pairs of images of various surface configurations. Plotting of data is possible in both cases. An interactive demo can also be used to explore the target image selection process

Additional Serine/Threonine Phosphorylation Reduces Binding Affinity but Preserves Interface Topography of Substrate Proteins to the c-Cbl TKB Domain

The E3-ubiquitin ligase, c-Cbl, is a multi-functional scaffolding protein that plays a pivotal role in controlling cell phenotype. As part of the ubiquitination and downregulation process, c-Cbl recognizes targets, such as tyrosine kinases and the Sprouty proteins, by binding to a conserved (NX/R)pY(S/T)XXP motif via its uniquely embedded SH2 domain (TKB domain). We previously outlined the mode of binding between the TKB domain and various substrate peptide motifs, including epidermal growth factor receptor (EGFR) and Sprouty2 (Spry2), and demonstrated that an intrapetidyl hydrogen bond forms between the (pY-1) arginine or (pY-2) asparagine and the phosphorylated tyrosine, which is crucial for binding. Recent reports demonstrated that, under certain types of stimulation, the serine/threonine residues at the pY+1 and/or pY+2 positions within this recognition motif of EGFR and Sprouty2 may be endogenously phosphorylated. Using structural and binding studies, we sought to determine whether this additional phosphorylation could affect the binding of the TKB domain to these peptides and consequently, whether the type of stimulation can dictate the degree to which substrates bind to c-Cbl. Here, we show that additional phosphorylation significantly reduces the binding affinity between the TKB domain and its target proteins, EGFR and Sprouty2, as compared to peptides bearing a single tyrosine phosphorylation. The crystal structure indicates that this is accomplished with minimal changes to the essential intrapeptidyl bond and that the reduced strength of the interaction is due to the charge repulsion between c-Cbl and the additional phosphate group. This obvious reduction in binding affinity, however, indicates that Cbl's interactions with its TKB-centered binding partners may be more favorable in the absence of Ser/Thr phosphorylation, which is stimulation and context specific in vivo. These results demonstrate the importance of understanding the environment in which certain residues are phosphorylated, and the necessity of including this in structural investigations

Solving the stereo correspondence problem with false matches.

The stereo correspondence problem exists because false matches between the images from multiple sensors camouflage the true (veridical) matches. True matches are correspondences between image points that have the same generative source; false matches are correspondences between similar image points that have different sources. This problem of selecting true matches among false ones must be overcome by both biological and artificial stereo systems in order for them to be useful depth sensors. The proposed re-examination of this fundamental issue shows that false matches form a symmetrical pattern in the array of all possible matches, with true matches forming the axis of symmetry. The patterning of false matches can therefore be used to locate true matches and derive the depth profile of the surface that gave rise to them. This reverses the traditional strategy, which treats false matches as noise. The new approach is particularly well-suited to extract the 3-D locations and shapes of camouflaged surfaces and to work in scenes characterized by high degrees of clutter. We demonstrate that the symmetry of false-match signals can be exploited to identify surfaces in random-dot stereograms. This strategy permits novel depth computations for target detection, localization, and identification by machine-vision systems, accounts for physiological and psychophysical findings that are otherwise puzzling and makes possible new ways for combining stereo and motion signals

Stereoacuity improves after short-term binocular pattern mismatch.

Monocular deprivation can chronically suppress vision in the deprived eye, if it is applied for sufficient duration early in life. However, it can have a transient effect in the opposite direction if applied briefly (a few hours) to normal adults. This short-term monocular deprivation appears to increase the gain for the deprived eye relative to the undeprived eye, affecting binocular vision by shifting the interocular balance. An interocular gain difference produced by patching a single eye might be expected to lower stereoacuity in normal observers, since raising contrast in just one eye reduces stereoacuity (the ‘contrast paradox’). We hypothesized that alternately depriving each eye in turn might benefit stereoacuity by increasing post‐deprivation gain in both eyes and dampening interocular suppression. We switched a translucent patch between the eyes of visually normal adult observers hourly for 6 hours. The unpatched eye viewed the natural visual environment. Compared to pre-patch performance, post-patch grating stereoacuity, measured during 20 minutes following patch removal, improved by 20% to 33%. In a second experiment, we alternately covered the left and right eyes of two observers with cylinder lenses to determine the existence of orientation-specific gain. Each eye was ‘patched’ with the lens for 45 or 60 minute periods, giving a total through-the-lens viewing of 4 to 6 hours. Unexpectedly, post-patching stereoacuity improved for test gratings with the same orientation as seen through the lens and worsened for orthogonal gratings. This result cannot be explained by monocular orientation adaptation. It implies that the interocular balance has a channel structure that is modulated not specifically by monocular deprivation but rather by interocular pattern mismatch. The post-patching enhancement evident in both experiments indicates that interocular suppression may limit stereoacuity under natural viewing conditions

Perceiving the stereo depth of simple stimuli isn\u27t simple: The case of gratings.

Horizontal disparities are directly linked to perceived stereo depth of two-dimensional stimuli but, surprisingly, the same does not hold for 1-D stimuli. 1-D stimuli, such as lines and gratings, have ambiguous disparity signals, an analog of the aperture problem in motion. One consequence is that the depth seen between two stimuli, one 1-D and the other 2-D, can vary with the orientation of the 1-D stimulus even if horizontal disparities remain unchanged. How relative disparities and orientations jointly affect the perceived depth between two 1-D stimuli is unknown. To determine the computation humans use, we had observers discriminate the depth order of a test grating presented in the context of a reference grating. Stimulus onsets were staggered over time. A reference grating was presented parafoveally, together with an identical fixation stimulus. After 1 second, a target grating was added to the display for 180 ms at the same eccentricity as the reference grating. Importantly, no other stimulus was available to mediate relative disparity calculations. We measured the disparity of the target grating required for the target and reference gratings to be seen at the same depth. We found that the size of this depth-matching disparity did not depend on horizontal disparity but instead was proportional to 1/cosine of the orientation difference between the two gratings. The sign of the depth-matching disparity varied with the reference grating\u27s clockwise versus counter-clockwise orientation relative to vertical. Cyclotorsion cannot account for the results; the largest possible ocular rotation would be much too small. Instead, the relative depth seen between the gratings is what would be expected from a normalization process that resolved the stereo aperture problem by notionally rotating the context stimuli to vertical, thus defining a standard functional disparity axis for computing relative disparities

Using the symmetry of false matches to solve the correspondence problem.

Sensing stereoscopic depth requires that image points be binocularly matched. Therein lies the correspondence problem: how are true matches distinguished from false ones? Conventional algorithms select true matches on the basis of correlated features and adherence to natural statistics, while rejecting false matches as noise. We propose here an alternative that uses the signals present in false matches to delineate the true solution. When visualized in a Keplerian array, binocular matches are symmetrically reflected about an axis that represents a potential solution. Surface properties such as slant and curvature are encoded the transformation that describes how one-half of the matches reflects across the symmetry axis onto the other. To implement this strategy, we convolved left and right images with Gaussian kernels of various standard deviations (spatial frequencies). We then produced Keplerian arrays by comparing filter responses across left and right spatial-frequency combinations. Responses that are minimally different across the eyes gave rise to regions of high symmetry; response position within the Keplerian array gave the location of a solution in space. Solutions possessing natural surface regularities consistently showed minimal differences for one left : right spatial frequency ratio, which is correlated with the local surface slant. As a result, combining responses within particular ratio families can distinguish true matches from false ones. True matches tend to be elongated and smoothly contoured, with symmetry preserved across all members of a ratio family from low to high spatial-frequency combinations. This approach is efficient; preprocessing is minimal since no feature extraction is involved. It can be implemented in machine vision to solve the correspondence problem for depth sensing algorithms. It is robust when tested against perfectly camouflaged surfaces in random dot stereograms and consistent with physiological data showing that false match signals are propagated to higher cortical areas along the dorsal pathway

Attention to pattern depth depends on pattern dimensionality

The perceived stereo depth separating two stimuli usually varies with the horizontal disparity difference between the stimuli. This, however, is not the case when one or both stimuli are one-dimensional. Instead, perceived depth depends on the difference between the disparity vectors of the two stimuli; relative disparity magnitude and direction both matter, interactively (Farell, Chai, Fernandez, Vis. Res., 2009). Here, we compare judgments of the depth of 1-D and 2-D stimuli, asking how attention affects this interaction. Our displays contained a central stimulus whose disparity varied across trials. This stimulus was either 1-D (a grating) or 2-D (a plaid). The stimuli surrounding the center were oblique-disparity plaids, the location of one being designated as relevant. The task was to judge the relative depth of the central stimulus and the relevant plaid; the remaining plaids were irrelevant throughout the block of trials and were to be ignored. Psychometric functions for depth judgments of the grating and the relevant plaid shifted laterally in response to the plaid\u27s disparity direction (parallel or orthogonal to the grating\u27s disparity). Interestingly, the disparities of irrelevant plaids produced exactly the same effect. Thus, attention failed to distinguish relevant and irrelevant stimuli when observers judged a grating-plaid pair. By contrast, when the stimuli being judged were both plaids, psychometric functions were affected neither by the disparities of irrelevant stimuli nor by the disparity direction of relevant stimuli. Attentional filtering of disparity signals thus succeeded only when observers judged the depths of 2-D stimuli. The judged depth of a 1-D stimulus varied with all the disparities in the display, whether relevant or irrelevant, revealing a disparity field that could be useful in transforming ambiguous 1-D component disparities into coherent object depths

Short-term monocular deprivation increases stereoacuity.

Monocular deprivation early in life can impair vision permanently (Hubel and Wiesel, 1970), but deprivation during adulthood has been thought to be without effect. However, adults who were monocularly deprived for a few hours show a short-term gain increase in the formerly deprived eye relative to the undeprived eye (Lunghi et al., 2011; Zhou et al., 2013). This binocular imbalance should affect stereoacuity. We previously showed that stereoacuity improves after depriving each eye sequentially, presumably by increasing the gain in both eyes. Following that argument, we hypothesize that: (1) monocular deprivation should lower stereoacuity (as does an interocular mismatch in stimulus contrast), but (2) depriving both eyes simultaneously should improve performance because balance is preserved as gain increases. We measured stereoacuity with a two-line depth discrimination test immediately before and after deprivation. Deprivation consisted of covering one or both eyes with a translucent patch for 2.5h, in accordance with previous studies. The patched eye received only low frequency and low contrast inputs while the unpatched eye viewed the natural environment directly. Observers were stereo-normal adults. Binocular deprivation improved stereoacuity by 35%; but contrary to our predictions, monocular deprivation also improved stereoacuity by 27-55%. Alternating the eye being deprived in three cycles during the 2.5h period also improved stereoacuity by 40%. Stereoacuity gradually returned to pre-deprivation levels after deprivation. Improved performance was correlated with duration; prolonging monocular deprivation to 4.5h produced further improvement (67%), but 45min had no observable effect. Stereoacuity improved regardless of the type of deprivation (alternating, binocular or monocular). Therefore, our results suggest that monocular contrast gain control was not responsible for the effects induced by deprivation. Rather, sustained balanced binocular input appears to lower the gain of stereo mechanisms through an adaptation effect. Deprivation avoids the adapting binocular interactions, and allows transient recovery