A Neural Model of Surface Perception: Lightness, Anchoring, and Filling-in

This article develops a neural model of how the visual system processes natural images under variable illumination conditions to generate surface lightness percepts. Previous models have clarified how the brain can compute the relative contrast of images from variably illuminate scenes. How the brain determines an absolute lightness scale that "anchors" percepts of surface lightness to us the full dynamic range of neurons remains an unsolved problem. Lightness anchoring properties include articulation, insulation, configuration, and are effects. The model quantatively simulates these and other lightness data such as discounting the illuminant, the double brilliant illusion, lightness constancy and contrast, Mondrian contrast constancy, and the Craik-O'Brien-Cornsweet illusion. The model also clarifies the functional significance for lightness perception of anatomical and neurophysiological data, including gain control at retinal photoreceptors, and spatioal contrast adaptation at the negative feedback circuit between the inner segment of photoreceptors and interacting horizontal cells. The model retina can hereby adjust its sensitivity to input intensities ranging from dim moonlight to dazzling sunlight. A later model cortical processing stages, boundary representations gate the filling-in of surface lightness via long-range horizontal connections. Variants of this filling-in mechanism run 100-1000 times faster than diffusion mechanisms of previous biological filling-in models, and shows how filling-in can occur at realistic speeds. A new anchoring mechanism called the Blurred-Highest-Luminance-As-White (BHLAW) rule helps simulate how surface lightness becomes sensitive to the spatial scale of objects in a scene. The model is also able to process natural images under variable lighting conditions.Air Force Office of Scientific Research (F49620-01-1-0397); Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); Office of Naval Research (N00014-01-1-0624

A detailed model of the primary visual pathway in the cat: comparison of afferent excitatory and intracortical inhibitory connection schemes for orientation selectivity

In order to arrive at a quantitative understanding of the dynamics of cortical neuronal networks, we simulated a detailed model of the primary visual pathway of the adult cat. This computer model comprises a 5 degrees x 5 degrees patch of the visual field at a retinal eccentricity of 4.5 degrees and includes 2048 ON- and OFF-center retinal beta-ganglion cells, 8192 geniculate X-cells, and 4096 simple cells in layer IV in area 17. The neurons are implemented as improved integrate-and-fire units. Cortical receptive fields are determined by the pattern of afferent convergence and by inhibitory intracortical connections. Orientation columns are implemented continuously with a realistic receptive field scatter and jitter in the preferred orientations. We first show that realistic ON-OFF-responses, orientation selectivity, velocity low-pass behaviour, null response, and responses to spot stimuli can be obtained with an appropriate alignment of geniculate neurons converging onto the cortical simple cell (Hubel and Wiesel, 1962) and in the absence of intracortical connections. However, the average receptive field elongation (length to width) required to obtain realistic orientation tuning is 4.0, much higher than the average observed elongation. This strongly argues for additional intracortical mechanisms sharpening orientation selectivity. In the second stage, we simulated five different inhibitory intracortical connection patterns (random, local, sparse-local, circular, and cross-orientation) in order to investigate the connection specificity necessary to achieve orientation tuning. Inhibitory connection schemes were superimposed onto Hubel and Wiesel-type receptive fields with an elongation of 1.78. Cross-orientation inhibition gave rise to different horizontal and vertical orientation tuning curves, something not observed experimentally. A combination of two inhibitory schemes, local and circular inhibition (a weak form of cross-orientation inhibition), is in good agreement with observed receptive field properties. The specificity required to establish these connections during development is low. We propose that orientation selectivity is caused by at least three different mechanisms (“eclectic” model): a weak afferent geniculate bias, broadly tuned cross-orientation inhibition, and some iso-orientation inhibition. The most surprising finding is that an isotropic connection scheme, circular inhibition, in which a cell inhibits all of its postsynaptic target cells at a distance of approximately 500 microns, enhances orientation tuning and leads to a significant directional bias. This is caused by the embedding of cortical cells within a columnar structure and does not depend on our specific assumptions

The context dependence of network response properties in the primary visual cortex of the primate and cat

In the mammalian visual system, stimulus context was investigated with respect to the ways it influenced neuronal mean response magnitude (the average number of spikes fired per second), response temporal structure (the timing of spikes with respect to one another), and the extent to which distributed neurones fired spikes synchronous due to synaptic interaction between them. Neurones were presented with bipartite grating stimuli, in which the spatio-temporal relationship between the grating activating the excitatory receptive field and that presented to the surrounding visual space could be varied systematically. Simultaneous extracellular recordings were made of the responses of up to four single neurones separated by 750-1000µm, in the lateral geniculate nucleus (LGN) of the thalamus in the cat, or the primary visual cortex (V1) of non-human primates or cats. Changing context systematically influenced the activity of groups of cells. The responses of 83% of primate V1 cells to discontinuous stimuli, in which the centre/surround orientation difference was greater than 45°, contained stronger oscillations at frequencies below 80Hz, than responses to continuous stimuli. Many cat and primate V1 neurones exhibited elevated response magnitudes to such stimuli. In primate V1, the strength of a cell's oscillatory discharge was dependent on stimulus configuration rather than response magnitude. In the LGN and V1, cell pairs with different orientation preferences fired synchronised responses when stimulated by specific discontinuous grating configurations. Stimulus specific synchronised LGN input, and reciprocal excitatory and inhibitory cortico-cortical connections could generate these properties of cells, and the network in which they exist. A model is proposed to account for the function significance of contour discontinuities in generating coherent neural representations of objects in the visual world. It involves response synchronisation in horizontal, feedforward and feedback interactions, within and between the LGN, V1, V2 and V4

Relationship between threshold and suprathreshold perception of position and stereoscopic depth.

We seek to determine the relationship between threshold and suprathreshold perception for position offset and stereoscopic depth perception under conditions that elevate their respective thresholds. Two threshold-elevating conditions were used: (1) increasing the interline gap and (2) dioptric blur. Although increasing the interline gap increases position (Vernier) offset and stereoscopic disparity thresholds substantially, the perception of suprathreshold position offset and stereoscopic depth remains unchanged. Perception of suprathreshold position offset also remains unchanged when the Vernier threshold is elevated by dioptric blur. We show that such normalization of suprathreshold position offset can be attributed to the topographical-map-based encoding of position. On the other hand, dioptric blur increases the stereoscopic disparity thresholds and reduces the perceived suprathreshold stereoscopic depth, which can be accounted for by a disparity-computation model in which the activities of absolute disparity encoders are multiplied by a Gaussian weighting function that is centered on the horopter. Overall, the statement equal suprathreshold perception occurs in threshold-elevated and unelevated conditions when the stimuli are equally above their corresponding thresholds describes the results better than the statement suprathreshold stimuli are perceived as equal when they are equal multiples of their respective threshold values

Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role

The lateral geniculate nucleus (LGN) has often been treated in the past as a linear filter that adds little to retinal processing of visual inputs. Here we review anatomical, neurophysiological, brain imaging, and modeling studies that have in recent years built up a much more complex view of LGN . These include effects related to nonlinear dendritic processing, cortical feedback, synchrony and oscillations across LGN populations, as well as involvement of LGN in higher level cognitive processing. Although recent studies have provided valuable insights into early visual processing including the role of LGN, a unified model of LGN responses to real-world objects has not yet been developed. In the light of recent data, we suggest that the role of LGN deserves more careful consideration in developing models of high-level visual processing

The Role of Non-Linearities in Visual Perception studied with a Computational Model of the Vertebrate Retina

Processing of visual stimuli in the vertebrate retina is complex and diverse. The retinal output to the higher centres of the nervous system, mediated by ganglion cells, consists of several different channels. Neurons in these channels can have very distinct response properties, which originate in different retinal pathways. In this work, the retinal origins and possible functional implications of the segregation of visual pathways will be investigated with a detailed, biologically realistic computational model of the retina. This investigation will focus on the two main retino-cortical pathways in the mammalian retina, the parvocellular and magnocellular systems, which are crucial for conscious visual perception. These pathways differ in two important aspects. The parvocellular system has a high spatial, but low temporal resolution. Conversely, the magnocellular system has a high temporal fidelity, spatial sampling however is less dense than for parvocellular cells. Additionally, the responses of magnocellular ganglion cells can show pronounced nonlinearities, while the parvocellular system is essentially linear. The origin of magnocellular nonlinearities is unknown and will be investigated in the first part of this work. As their main source, the results suggest specific properties of the photoreceptor response and a specialised amacrine cell circuit in the inner retina. The results further show that their effect combines in a multiplicative way. The model is then used to examine the influence of nonlinearities on the responses of ganglion cells in the presence of involuntary fixational eye movements. Two different stimulus conditions will be considered: visual hyperacuity and motion induced illusions. In both cases, it is possible to directly compare properties of the ganglion cell population response with psychophysical data, which allows for an analysis of the influence of different components of the retinal circuitry. The simulation results suggest an important role for nonlinearities in the magnocellular stream for visual perception in both cases. First, it will be shown how nonlinearities, triggered by fixational eye movements, can strongly enhance the spatial precision of magnocellular ganglion cells. As a result, their performance in a hyperacuity task can be equal to or even surpass that of the parvocellular system. Second, the simulations imply that the origin of some of the illusory percepts elicited by fixational eye movements could be traced back to the nonlinear properties of magnocellular ganglion cells. As these activity patterns strongly differ from those in the parvocellular system, it appears that the magnocellular system can strongly dominate visual perception in certain conditions. Taken together, the results of this theoretical study suggest that retinal nonlinearities may be important for and strongly influence visual perception. The model makes several experimentally verifiable predictions to further test and quantify these findings. Furthermore, models investigating higher visual processing stages may benefit from this work, which could provide the basis to produce realistic afferent input

Computational role of disinhibition in brain function

Neurons are connected to form functional networks in the brain. When neurons are combined in sequence, nontrivial effects arise. One example is disinhibition; that is, inhibition to another inhibitory factor. Disinhibition may be serving an important purpose because a large number of local circuits in the brain contain disinhibitory connections. However, their exact functional role is not well understood. The objective of this dissertation is to analyze the computational role of disinhibition in brain function, especially in visual perception and attentional control. My approach is to propose computational models of disinhibition and then map the model to the local circuits in the brain to explain psychological phenomena. Several computational models are proposed in this dissertation to account for disinhibition. (1) A static inverse difference of Gaussian filter (IDoG) is derived to account explicitly for the spatial effects of disinhibition. IDoG can explain a number of complex brightness-contrast illusions, such as the periphery problem in the Hermann grid and the White's effect. The IDoG model can also be used to explain orientation perception of multiple lines as in the modified version of Poggendorff illusion. (2) A spatio-temporal model (IDoGS) in early vision is derived and it successfully explains the scintillating grid illusion, which is a stationary display giving rise to a striking, dynamic, scintillating effect. (3) An interconnected Cohen-Grossberg neural network model (iCGNN) is proposed to address the dynamics of disinhibitory neural networks with a layered structure. I derive a set of sufficient conditions for such an interconnected system to reach asymptotic stability. (4) A computational model combining recurrent and feed-forward disinhibition is designed to account for input-modulation in temporal selective attention. The main contribution of this research is that it developed a unified framework of disinhibition to model several different kinds of neural circuits to account for various perceptual and attentional phenomena. Investigating the role of disinhibition in the brain can provide us with a deeper understanding of how the brain can give rise to intelligent and complex functions

Development and encoding of visual statistics in the primary visual cortex

How do circuits in the mammalian cerebral cortex encode properties of the sensory environment in a way that can drive adaptive behavior? This question is fundamental to neuroscience, but it has been very difficult to approach directly. Various computational and theoretical models can explain a wide range of phenomena observed in the primary visual cortex (V1), including the anatomical organization of its circuits, the development of functional properties like orientation tuning, and behavioral effects like surround modulation. However, so far no model has been able to bridge these levels of description to explain how the machinery that develops directly affects behavior. Bridging these levels is important, because phenomena at any one specific level can have many possible explanations, but there are far fewer possibilities to consider once all of the available evidence is taken into account. In this thesis we integrate the information gleaned about cortical development, circuit and cell-type specific interactions, and anatomical, behavioral and electrophysiological measurements, to develop a computational model of V1 that is constrained enough to make predictions across multiple levels of description. Through a series of models incorporating increasing levels of biophysical detail and becoming increasingly better constrained, we are able to make detailed predictions for the types of mechanistic interactions required for robust development of cortical maps that have a realistic anatomical organization, and thereby gain insight into the computations performed by the primary visual cortex. The initial models focus on how existing anatomical and electrophysiological knowledge can be integrated into previously abstract models to give a well-grounded and highly constrained account of the emergence of pattern-specific tuning in the primary visual cortex. More detailed models then address the interactions between specific excitatory and inhibitory cell classes in V1, and what role each cell type may play during development and function. Finally, we demonstrate how these cell classes come together to form a circuit that gives rise not only to robust development but also the development of realistic lateral connectivity patterns. Crucially, these patterns reflect the statistics of the visual environment to which the model was exposed during development. This property allows us to explore how the model is able to capture higher-order information about the environment and use that information to optimize neural coding and aid the processing of complex visual tasks. Using this model we can make a number of very specific predictions about the mechanistic workings of the brain. Specifically, the model predicts a crucial role of parvalbumin-expressing interneurons in robust development and divisive normalization, while it implicates somatostatin immunoreactive neurons in mediating longer range and feature-selective suppression. The model also makes predictions about the role of these cell classes in efficient neural coding and under what conditions the model fails to organize. In particular, we show that a tight coupling of activity between the principal excitatory population and the parvalbumin population is central to robust and stable responses and organization, which may have implications for a variety of diseases where parvalbumin interneuron function is impaired, such as schizophrenia and autism. Further the model explains the switch from facilitatory to suppressive surround modulation effects as a simple by-product of the facilitating response function of long-range excitatory connections targeting a specialized class of inhibitory interneurons. Finally, the model allows us to make predictions about the statistics that are encoded in the extensive network of long-range intra-areal connectivity in V1, suggesting that even V1 can capture high-level statistical dependencies in the visual environment. The final model represents a comprehensive and well constrained model of the primary visual cortex, which for the first time can relate the physiological properties of individual cell classes to their role in development, learning and function. While the model is specifically tuned for V1, all mechanisms introduced are completely general, and can be used as a general cortical model, useful for studying phenomena across the visual cortex and even the cortex as a whole. This work is also highly relevant for clinical neuroscience, as the cell types studied here have been implicated in neurological disorders as wide ranging as autism, schizophrenia and Parkinson’s disease

Interplay between Primary Cortical Areas and Crossmodal Plasticity

Perceptual representations are built through multisensory interactions underpinned by dense anatomical and functional neural networks that interconnect primary and associative cortical areas. There is compelling evidence that primary sensory cortical areas do not work in segregation, but play a role in early processes of multisensory integration. In this chapter, we firstly review previous and recent literature showing how multimodal interactions between primary cortices may contribute to refining perceptual representations. Secondly, we discuss findings providing evidence that, following peripheral damage to a sensory system, multimodal integration may promote sensory substitution in deprived cortical areas and favor compensatory plasticity in the spared sensory cortices