A Nonlinear Model of Spatiotemporal Retinal Processing: Simulations of X and Y Retinal Ganglion Cell Behavior

This article describes a nonlinear model of neural processing in the vertebrate retina, comprising model photoreceptors, model push-pull bipolar cells, and model ganglion cells. Previous analyses and simulations have shown that with a choice of parameters that mimics beta cells, the model exhibits X-like linear spatial summation (null response to contrast-reversed gratings) in spite of photoreceptor nonlinearities; on the other hand, a choice of parameters that mimics alpha cells leads to Y-like frequency doubling. This article extends the previous work by showing that the model can replicate qualitatively many of the original findings on X and Y cells with a fixed choice of parameters. The results generally support the hypothesis that X and Y cells can be seen as functional variants of a single neural circuit. The model also suggests that both depolarizing and hyperpolarizing bipolar cells converge onto both ON and OFF ganglion cell types. The push-pull connectivity enables ganglion cells to remain sensitive to deviations about the mean output level of nonlinear photoreceptors. These and other properties of the push-pull model are discussed in the general context of retinal processing of spatiotemporal luminance patterns.Alfred P. Sloan Research Fellowship (BR-3122); Air Force Office of Scientific Research (F49620-92-J-0499

Motion clouds: model-based stimulus synthesis of natural-like random textures for the study of motion perception

Choosing an appropriate set of stimuli is essential to characterize the response of a sensory system to a particular functional dimension, such as the eye movement following the motion of a visual scene. Here, we describe a framework to generate random texture movies with controlled information content, i.e., Motion Clouds. These stimuli are defined using a generative model that is based on controlled experimental parametrization. We show that Motion Clouds correspond to dense mixing of localized moving gratings with random positions. Their global envelope is similar to natural-like stimulation with an approximate full-field translation corresponding to a retinal slip. We describe the construction of these stimuli mathematically and propose an open-source Python-based implementation. Examples of the use of this framework are shown. We also propose extensions to other modalities such as color vision, touch, and audition

Impact of the pulvinar on the ventral pathway of the cat visual cortex

Signals from the retina are relayed to the lateral geniculate nucleus from which they are sent to the primary visual cortex. At the cortical level, the information is transferred across several visual areas in which the complexity of the processing increases progressively. Anatomical and functional evidence demonstrate the existence of two main pathways in visual cortex processing distinct features of the visual information: the dorsal and ventral streams. Cortical areas composing the dorsal stream are implicated mostly in motion processing while those comprising the ventral stream are involved in the processing of form and colour. This classic view of the cortical functional organization is challenged by the existence of reciprocal connections of visual cortical areas with the thalamic nucleus named pulvinar. These connections allow the creation of a trans-thalamic pathway that parallels the cortico-cortical communications across the visual hierarchy. The main goal of the present thesis is twofold: first, to obtain a better comprehension of the processing of light increments and decrements in an area of the cat ventral stream (area 21a); second, to characterize the nature of the thalamo-cortical inputs from the cat lateral posterior nucleus (LP) to area 21a. In study #1, we investigated the spatiotemporal response profile of neurons from area 21a to light increments (brights) and decrements (darks) using a reverse correlation analysis of a sparse noise stimulus. Our findings showed that 21a neurons exhibited stronger responses to darks with receptive fields exhibiting larger dark subfields. However, no differences were found between the temporal dynamics of brights and darks. In comparison with the primary visual cortex, the dark preference in area 21a was found to be strongly enhanced, supporting the notion that the asymmetries between brights and darks are transmitted and amplified along the ventral stream. In study #2, we investigated the impact of the reversible pharmacological inactivation of the LP nucleus on the contrast response function (CRF) of neurons from area 21a and the primary visual cortex (area 17). The thalamic inactivation yielded distinct effects on both cortical areas. While in area 17 the LP inactivation caused a slight decrease in the response gain, in area 21a a strong increase was observed. Thus, our findings suggest that the LP exerts a modulatory influence on the cortical processing along the ventral stream with stronger impact on higher order extrastriate areas. Taken together, our findings allowed a better comprehension of the functional properties of the cat ventral stream and contributed to the current knowledge on the role of the pulvinar on the cortico-thalamo-cortical processing of visual information.Les signaux provenant de la rétine sont relayés dans le corps géniculé latéral où ils sont envoyés au cortex visuel primaire. L’information passe ensuite à travers plusieurs aires visuelles où la complexité du traitement augmente progressivement. Des données tant anatomiques que fonctionnelles ont démontré l’existence de deux voies principales qui traitent différentes propriétés de l’information visuelle : les voies dorsale et ventrale. Les aires corticales composant la voie dorsale sont impliquées principalement dans le traitement du mouvement tandis que les aires de la voie ventrale sont impliquées dans le traitement de la forme et de la couleur. Cette vision classique de l’organisation fonctionnelle du cortex est toutefois remise en question par l’existence de connections réciproques entre les aires corticales visuelles et le pulvinar, un noyau thalamique. En effet, ces connections permettent la création d’une voie trans-thalamique parallèle aux connections cortico-corticales à travers la hiérarchie visuelle. Le but principal de la présente thèse consiste en deux volets : le premier est d’obtenir une meilleure compréhension du traitement des incréments et décréments de la lumière dans une aire de la voie ventrale du chat (aire 21a); le second est de caractériser la nature des inputs thalamo-corticaux du noyau latéral postérieur (LP) à l’aire 21a chez le chat. Dans l’étude #1, nous avons investigué le profil spatiotemporel des réponses des neurones de l’aire 21a aux incréments (blancs) et décréments (noirs) de lumière en utilisant l’analyse de corrélation inverse d’un stimulus de bruit épars. Les neurones de l’aire 21a ont répondu plus fortement aux stimuli noirs, en montrant des champs récepteurs avec des sous-champs noirs plus larges. Cependant, aucune différence n’a été trouvée en ce qui concerne les dynamiques temporelles des réponses aux blancs et aux noirs. En comparaison avec le cortex visuel primaire, la préférence aux stimuli noirs dans l’aire 21a s’est avérée fortement augmentée. Ces données indiquent que les asymétries entre les réponses aux blancs et aux noirs sont transmises et amplifiées à travers la voie ventrale. Dans l’étude #2, nous avons investigué l’impact de l’inactivation pharmacologique réversible du noyau LP sur la fonction de réponse au contraste (CRF) des neurones de l’aire 21a et du cortex visuel primaire (aire 17). L’inactivation a eu différents effets dans les deux aires corticales. Alors que, dans l’aire 17, l’inactivation du LP a causé une légère réduction du gain de la réponse, une forte augmentation a été observée dans l’aire 21a. Ainsi, nos résultats suggèrent que le LP exerce une influence modulatrice dans le traitement cortical à travers la voie ventrale avec un impact plus important dans des aires extrastriées de plus haut niveau. Nos résultats ont permis d’avoir une meilleure compréhension des propriétés fonctionnelles de la voie ventrale du chat et de contribuer à enrichir les connaissances actuelles sur le rôle du pulvinar dans le traitement cortico-thalamo-cortical de l’information visuelle

The Role of Non-Linearities in Visual Perception studied with a Computational Model of the Vertebrate Retina

Processing of visual stimuli in the vertebrate retina is complex and diverse. The retinal output to the higher centres of the nervous system, mediated by ganglion cells, consists of several different channels. Neurons in these channels can have very distinct response properties, which originate in different retinal pathways. In this work, the retinal origins and possible functional implications of the segregation of visual pathways will be investigated with a detailed, biologically realistic computational model of the retina. This investigation will focus on the two main retino-cortical pathways in the mammalian retina, the parvocellular and magnocellular systems, which are crucial for conscious visual perception. These pathways differ in two important aspects. The parvocellular system has a high spatial, but low temporal resolution. Conversely, the magnocellular system has a high temporal fidelity, spatial sampling however is less dense than for parvocellular cells. Additionally, the responses of magnocellular ganglion cells can show pronounced nonlinearities, while the parvocellular system is essentially linear. The origin of magnocellular nonlinearities is unknown and will be investigated in the first part of this work. As their main source, the results suggest specific properties of the photoreceptor response and a specialised amacrine cell circuit in the inner retina. The results further show that their effect combines in a multiplicative way. The model is then used to examine the influence of nonlinearities on the responses of ganglion cells in the presence of involuntary fixational eye movements. Two different stimulus conditions will be considered: visual hyperacuity and motion induced illusions. In both cases, it is possible to directly compare properties of the ganglion cell population response with psychophysical data, which allows for an analysis of the influence of different components of the retinal circuitry. The simulation results suggest an important role for nonlinearities in the magnocellular stream for visual perception in both cases. First, it will be shown how nonlinearities, triggered by fixational eye movements, can strongly enhance the spatial precision of magnocellular ganglion cells. As a result, their performance in a hyperacuity task can be equal to or even surpass that of the parvocellular system. Second, the simulations imply that the origin of some of the illusory percepts elicited by fixational eye movements could be traced back to the nonlinear properties of magnocellular ganglion cells. As these activity patterns strongly differ from those in the parvocellular system, it appears that the magnocellular system can strongly dominate visual perception in certain conditions. Taken together, the results of this theoretical study suggest that retinal nonlinearities may be important for and strongly influence visual perception. The model makes several experimentally verifiable predictions to further test and quantify these findings. Furthermore, models investigating higher visual processing stages may benefit from this work, which could provide the basis to produce realistic afferent input

Towards building a more complex view of the lateral geniculate nucleus: Recent advances in understanding its role

The lateral geniculate nucleus (LGN) has often been treated in the past as a linear filter that adds little to retinal processing of visual inputs. Here we review anatomical, neurophysiological, brain imaging, and modeling studies that have in recent years built up a much more complex view of LGN . These include effects related to nonlinear dendritic processing, cortical feedback, synchrony and oscillations across LGN populations, as well as involvement of LGN in higher level cognitive processing. Although recent studies have provided valuable insights into early visual processing including the role of LGN, a unified model of LGN responses to real-world objects has not yet been developed. In the light of recent data, we suggest that the role of LGN deserves more careful consideration in developing models of high-level visual processing

The role of spatial frequency information for ERP components sensitive to faces and emotional facial expression

To investigate the impact of spatial frequency on emotional facial expression analysis, ERPs were recorded in response to low spatial frequency (LSF), high spatial frequency (HSF), and unfiltered broad spatial frequency (BSF) faces with fearful or neutral expressions, houses, and chairs. In line with previous findings, BSF fearful facial expressions elicited a greater frontal positivity than BSF neutral facial expressions, starting at about 150 ms after stimulus onset. In contrast, this emotional expression effect was absent for HSF and LSF faces. Given that some brain regions involved in emotion processing, such as amygdala and connected structures, are selectively tuned to LSF visual inputs, these data suggest that ERP effects of emotional facial expression do not directly reflect activity in these regions. It is argued that higher order neocortical brain systems are involved in the generation of emotion-specific waveform modulations. The face-sensitive N170 component was neither affected by emotional facial expression nor by spatial frequency information

The Role of Early Recurrence in Improving Visual Representations

This dissertation proposes a computational model of early vision with recurrence, termed as early recurrence. The idea is motivated from the research of the primate vision. Specifically, the proposed model relies on the following four observations. 1) The primate visual system includes two main visual pathways: the dorsal pathway and the ventral pathway; 2) The two pathways respond to different visual features; 3) The neurons of the dorsal pathway conduct visual information faster than that of the neurons of the ventral pathway; 4) There are lower-level feedback connections from the dorsal pathway to the ventral pathway. As such, the primate visual system may implement a recurrent mechanism to improve visual representations of the ventral pathway. Our work starts from a comprehensive review of the literature, based on which a conceptualization of early recurrence is proposed. Early recurrence manifests itself as a form of surround suppression. We propose that early recurrence is capable of refining the ventral processing using results of the dorsal processing. Our work further defines a set of computational components to formalize early recurrence. Although we do not intend to model the true nature of biology, to verify that the proposed computation is biologically consistent, we have applied the model to simulate a neurophysiological experiment of a bar-and-checkerboard and a psychological experiment involving a moving contour illusion. Simulation results indicated that the proposed computation behaviourally reproduces the original observations. The ultimate goal of this work is to investigate whether the proposal is capable of improving computer vision applications. To do this, we have applied the model to a variety of applications, including visual saliency and contour detection. Based on comparisons against the state-of-the-art, we conclude that the proposed model of early recurrence sheds light on a generally applicable yet lightweight approach to boost real-life application performance

A combined experimental and computational approach to investigate emergent network dynamics based on large-scale neuronal recordings

Sviluppo di un approccio integrato computazionale-sperimentale per lo studio di reti neuronali mediante registrazioni elettrofisiologich

Suboptimal eye movements for seeing fine details.

Human eyes are never stable, even during attempts of maintaining gaze on a visual target. Considering transient response characteristics of retinal ganglion cells, a certain amount of motion of the eyes is required to efficiently encode information and to prevent neural adaptation. However, excessive motion of the eyes leads to insufficient exposure to the stimuli, which creates blur and reduces visual acuity. Normal miniature eye movements fall in between these extremes, but it is unclear if they are optimally tuned for seeing fine spatial details. We used a state-of-the-art retinal imaging technique with eye tracking to address this question. We sought to determine the optimal gain (stimulus/eye motion ratio) that corresponds to maximum performance in an orientation-discrimination task performed at the fovea. We found that miniature eye movements are tuned but may not be optimal for seeing fine spatial details