From Stereogram to Surface: How the Brain Sees the World in Depth

When we look at a scene, how do we consciously see surfaces infused with lightness and color at the correct depths? Random Dot Stereograms (RDS) probe how binocular disparity between the two eyes can generate such conscious surface percepts. Dense RDS do so despite the fact that they include multiple false binocular matches. Sparse stereograms do so even across large contrast-free regions with no binocular matches. Stereograms that define occluding and occluded surfaces lead to surface percepts wherein partially occluded textured surfaces are completed behind occluding textured surfaces at a spatial scale much larger than that of the texture elements themselves. Earlier models suggest how the brain detects binocular disparity, but not how RDS generate conscious percepts of 3D surfaces. A neural model predicts how the layered circuits of visual cortex generate these 3D surface percepts using interactions between visual boundary and surface representations that obey complementary computational rules.Air Force Office of Scientific Research (F49620-01-1-0397); National Science Foundation (EIA-01-30851, SBE-0354378); Office of Naval Research (N00014-01-1-0624

How Does the Cerebral Cortex Work? Developement, Learning, Attention, and 3D Vision by Laminar Circuits of Visual Cortex

A key goal of behavioral and cognitive neuroscience is to link brain mechanisms to behavioral functions. The present article describes recent progress towards explaining how the visual cortex sees. Visual cortex, like many parts of perceptual and cognitive neocortex, is organized into six main layers of cells, as well as characteristic sub-lamina. Here it is proposed how these layered circuits help to realize the processes of developement, learning, perceptual grouping, attention, and 3D vision through a combination of bottom-up, horizontal, and top-down interactions. A key theme is that the mechanisms which enable developement and learning to occur in a stable way imply properties of adult behavior. These results thus begin to unify three fields: infant cortical developement, adult cortical neurophysiology and anatomy, and adult visual perception. The identified cortical mechanisms promise to generalize to explain how other perceptual and cognitive processes work.Air Force Office of Scientific Research (F49620-01-1-0397); Office of Naval Research (N00014-01-1-0624

On the Inverse Problem of Binocular 3D Motion Perception

It is shown that existing processing schemes of 3D motion perception such as interocular velocity difference, changing disparity over time, as well as joint encoding of motion and disparity, do not offer a general solution to the inverse optics problem of local binocular 3D motion. Instead we suggest that local velocity constraints in combination with binocular disparity and other depth cues provide a more flexible framework for the solution of the inverse problem. In the context of the aperture problem we derive predictions from two plausible default strategies: (1) the vector normal prefers slow motion in 3D whereas (2) the cyclopean average is based on slow motion in 2D. Predicting perceived motion directions for ambiguous line motion provides an opportunity to distinguish between these strategies of 3D motion processing. Our theoretical results suggest that velocity constraints and disparity from feature tracking are needed to solve the inverse problem of 3D motion perception. It seems plausible that motion and disparity input is processed in parallel and integrated late in the visual processing hierarchy

The neurophysiology of stereoscopic vision

PhD ThesisMany animals are able to perceive stereoscopic depth owing to the disparity information that arises from the left and right eyes' horizontal displacement on the head. The initial computation of disparity happens in primary visual cortex (V1) and is largely considered to be a correlation-based computation. In other words, the computational role of V1 as it pertains to stereoscopic vision can be seen to roughly perform a binocular cross-correlation between the images of the left and right eyes. This view is based on the unique success of a correlation-based model of disparity-selective cells { the binocular energy model (BEM). This thesis addresses two unresolved challenges to this narrative. First, recent evidence suggests that a correlation-based view of primary visual cortex is unable to account for human perception of depth in a stimulus where the binocular correlation is on average zero. Chapters 1 and 2 show how a simple extension of the BEM which better captures key properties of V1 neurons allows model cells to signal depth in such stimuli. We also build a psychophysical model which captures human performance closely, and recording from V1 in the macaque, we then show that these predicted properties are indeed observed in real V1 neurons. The second challenge relates to the long-standing inability of the BEM to capture responses to anticorrelated stimuli: stimuli where the contrast is reversed in the two eyes (e.g. black features in the left eye are matched with identical white features in the right eye). Real neurons respond less strongly to these stimuli than model cells. In Chapter 3 and 4, we make use of recent advances in optimisation routines and exhaustively test the ability of a generalised BEM to capture this property. We show that even the best- tting generalised BEM units only go some way towards describing neuronal responses. This is the rst exhaustive empirical test of this in uential modelling framework, and we speculate on what is needed to develop a more complete computational account of visual processing in primary visual cortex

Boundary, Brightness, and Depth Interactions During Preattentive Representation and Attentive Recognition of Figure and Ground

This article applies a recent theory of 3-D biological vision, called FACADE Theory, to explain several percepts which Kanizsa pioneered. These include 3-D pop-out of an occluding form in front of an occluded form, leading to completion and recognition of the occluded form; 3-D transparent and opaque percepts of Kanizsa squares, with and without Varin wedges; and interactions between percepts of illusory contours, brightness, and depth in response to 2-D Kanizsa images. These explanations clarify how a partially occluded object representation can be completed for purposes of object recognition, without the completed part of the representation necessarily being seen. The theory traces these percepts to neural mechanisms that compensate for measurement uncertainty and complementarity at individual cortical processing stages by using parallel and hierarchical interactions among several cortical processing stages. These interactions are modelled by a Boundary Contour System (BCS) that generates emergent boundary segmentations and a complementary Feature Contour System (FCS) that fills-in surface representations of brightness, color, and depth. The BCS and FCS interact reciprocally with an Object Recognition System (ORS) that binds BCS boundary and FCS surface representations into attentive object representations. The BCS models the parvocellular LGN→Interblob→Interstripe→V4 cortical processing stream, the FCS models the parvocellular LGN→Blob→Thin Stripe→V4 cortical processing stream, and the ORS models inferotemporal cortex.Air Force Office of Scientific Research (F49620-92-J-0499); Defense Advanced Research Projects Agency (N00014-92-J-4015); Office of Naval Research (N00014-91-J-4100

Towards a Unified Theory of Neocortex: Laminar Cortical Circuits for Vision and Cognition

A key goal of computational neuroscience is to link brain mechanisms to behavioral functions. The present article describes recent progress towards explaining how laminar neocortical circuits give rise to biological intelligence. These circuits embody two new and revolutionary computational paradigms: Complementary Computing and Laminar Computing. Circuit properties include a novel synthesis of feedforward and feedback processing, of digital and analog processing, and of pre-attentive and attentive processing. This synthesis clarifies the appeal of Bayesian approaches but has a far greater predictive range that naturally extends to self-organizing processes. Examples from vision and cognition are summarized. A LAMINART architecture unifies properties of visual development, learning, perceptual grouping, attention, and 3D vision. A key modeling theme is that the mechanisms which enable development and learning to occur in a stable way imply properties of adult behavior. It is noted how higher-order attentional constraints can influence multiple cortical regions, and how spatial and object attention work together to learn view-invariant object categories. In particular, a form-fitting spatial attentional shroud can allow an emerging view-invariant object category to remain active while multiple view categories are associated with it during sequences of saccadic eye movements. Finally, the chapter summarizes recent work on the LIST PARSE model of cognitive information processing by the laminar circuits of prefrontal cortex. LIST PARSE models the short-term storage of event sequences in working memory, their unitization through learning into sequence, or list, chunks, and their read-out in planned sequential performance that is under volitional control. LIST PARSE provides a laminar embodiment of Item and Order working memories, also called Competitive Queuing models, that have been supported by both psychophysical and neurobiological data. These examples show how variations of a common laminar cortical design can embody properties of visual and cognitive intelligence that seem, at least on the surface, to be mechanistically unrelated.National Science Foundation (SBE-0354378); Office of Naval Research (N00014-01-1-0624

Binocular integration using stereo motion cues to drive behavior in mice

The visual system presents an opportunity to study how two signals converge to generate a novel representation of the world: depth. The slight difference in positions between the two eyes means that different images are encoded by the left and right eyes by generating disparity signals. Another way to generate depth signals is by presenting different motion signals to the two eyes. Even though the binocular visual system has been studied for a long time, the mechanisms behind binocular integration when objects move in depth are largely unknown. In this dissertation, I demonstrate a new model for studying motion-in-depth signals using mice. Mice are an attractive animal to study the binocular visual system not only because they share common visual pathway as primates and other mammals, but also because there are genetic tools that can be used to study the underlying circuitry for binocular integration during motion-in-depth cues. Thus far there have been very few studies regarding binocularity in mice. This dissertation will focus on the behavioral output during stereoscopic motion-in-depth signals in mice and investigate visual areas involved in these behaviors. In the first section, I investigate whether mice discriminate motion-in-depth signals like primates, using disparity and motion signals presented to each eye. I find that mice are able to discriminate towards and away stimuli and that the binocular neurons in the visual cortex were critical for the computation of this signal. In the second section we measured optokinetic eye movement generated by motion-in-depth stimulus. I found that vergence eye movement in mice is driven primarily by the motion signals presented in each eye. This phenomenon can be explained largely by the summation of monocular motor signals of the two eyes that happens subcortically. These two experiments both show clear behavioral output that can be only generated when presented with binocular motion-in-depth signals. I find both cortical and subcortical components of binocular integration that are responsible for the generation of these behavior outputs which demonstrates the complicated nature of binocular integration associated with motion-in-depth signals. My work in this dissertation provides the foundation for studying binocular integration in rodentsNeuroscienc