Contour Integration over Time: Psychophysical and fMRI Evidence.

The brain integrates discrete but collinear stimuli to perceive global contours. Previous contour integration (CI) studies mainly focus on integration over space, and CI is attributed to either V1 long-range connections or contour processing in high-visual areas that top-down modulate V1 responses. Here, we show that CI also occurs over time in a design that minimizes the roles of V1 long-range interactions. We use tilted contours embedded in random orientation noise and moving horizontally behind a fixed vertical slit. Individual contour elements traveling up/down within the slit would be encoded over time by parallel, rather than aligned, V1 neurons. However, we find robust contour detection even when the slit permits only one viewable contour element. Similar to CI over space, CI over time also obeys the rule of collinearity. fMRI evidence shows that while CI over space engages visual areas as early as V1, CI over time mainly engages higher dorsal and ventral visual areas involved in shape processing, as well as posterior parietal regions involved in visual memory that can represent the orientation of temporally integrated contours. These results suggest at least partially dissociable mechanisms for implementing the Gestalt rule of continuity in CI over space and time.European Community’s Seventh Framework Programme (FP7/2007-2013) (Grant ID: 255577), Biotechnology and Biological Sciences Research Council (Grant IDs: D52199X , E027436), National Natural Science Foundation of China (Grant IDs: 31230030, 31571160, 91432102

Contour Integration over Time: Psychophysical and fMRI Evidence.

The brain integrates discrete but collinear stimuli to perceive global contours. Previous contour integration (CI) studies mainly focus on integration over space, and CI is attributed to either V1 long-range connections or contour processing in high-visual areas that top-down modulate V1 responses. Here, we show that CI also occurs over time in a design that minimizes the roles of V1 long-range interactions. We use tilted contours embedded in random orientation noise and moving horizontally behind a fixed vertical slit. Individual contour elements traveling up/down within the slit would be encoded over time by parallel, rather than aligned, V1 neurons. However, we find robust contour detection even when the slit permits only one viewable contour element. Similar to CI over space, CI over time also obeys the rule of collinearity. fMRI evidence shows that while CI over space engages visual areas as early as V1, CI over time mainly engages higher dorsal and ventral visual areas involved in shape processing, as well as posterior parietal regions involved in visual memory that can represent the orientation of temporally integrated contours. These results suggest at least partially dissociable mechanisms for implementing the Gestalt rule of continuity in CI over space and time

Disambiguating Multi–Modal Scene Representations Using Perceptual Grouping Constraints

In its early stages, the visual system suffers from a lot of ambiguity and noise that severely limits the performance of early vision algorithms. This article presents feedback mechanisms between early visual processes, such as perceptual grouping, stereopsis and depth reconstruction, that allow the system to reduce this ambiguity and improve early representation of visual information. In the first part, the article proposes a local perceptual grouping algorithm that — in addition to commonly used geometric information — makes use of a novel multi–modal measure between local edge/line features. The grouping information is then used to: 1) disambiguate stereopsis by enforcing that stereo matches preserve groups; and 2) correct the reconstruction error due to the image pixel sampling using a linear interpolation over the groups. The integration of mutual feedback between early vision processes is shown to reduce considerably ambiguity and noise without the need for global constraints

Models of learning in the visual system: dependence on retinal eccentricity

In the primary visual cortex of primates relatively more space is devoted to the representation of the central visual field in comparison to the representation of the peripheral visual field. Experimentally testable theories about the factors and mechanisms which may have determined this inhomogeneous mapping may provide valuable insights into general processing principles in the visual system. Therefore, I investigated to which visual situations this inhomogeneous representation of the visual field is well adapted, and which mechanisms could support its refinement and stabilization during individual development. Furthermore, I studied possible functional consequences of the inhomogeneous representation for visual processing at central and peripheral locations of the visual field. Vision plays an important role during navigation. Thus, visual processing should be well adapted to self-motion. Therefore, I assumed that spatially inhomogeneous retinal velocity distributions, caused by static objects during self-motion along the direction of gaze, are transformed on average into spatially homogeneous cortical velocity distributions. This would have the advantage that the cortical mechanisms, concerned with the processing of self-motion, can be identical in their spatial and temporal properties across the representation of the whole visual field. This is the case if the arrangement of objects relative to the observer corresponds to an ellipsoid with the observer in its center. I used the resulting flow field to train a network model of pulse coding neurons with a Hebbian learning rule. The distribution of the learned receptive fields is in agreement with the inhomogeneous cortical representation of the visual field. These results suggest that self motion may have played an important role in the evolution of the visual system and that the inhomogeneous cortical representation of the visual field can be refined and stabilized by Hebbian learning mechanisms during ontogenesis under natural viewing conditions. In addition to the processing of self-motion, an important task of the visual system is the grouping and segregation of local features within a visual scene into coherent objects. Therefore, I asked how the corresponding mechanisms depend on the represented position of the visual field. It is assumed that neuronal connections within the primary visual cortex subserve this grouping process. These connections develop after eye-opening in dependence on the visual input. How does the lateral connectivity depend on the represented position of the visual field? With increasing eccentricity, primary cortical receptive fields become larger and the cortical magnification of the visual field declines. Therefore, I investigated the spatial statistics of real-world scenes with respect to the spatial filter-properties of cortical neurons at different locations of the visual field. I show that correlations between collinearly arranged filters of the same size and orientation increase with increasing filter size. However, in distances relative to the size of the filters, collinear correlations decline more steeply with increasing distance for larger filters. This provides evidence against a homogeneous cortical connectivity across the whole visual field with respect to the coding of spatial object properties. Two major retino-cortical pathways are the magnocellular (M) and the parvocellular (P) pathways. While neurons along the M-pathway display temporal bandpass characteristics, neurons along the P-pathway show temporal lowpass characteristics. The ratio of P- to M-cells is not constant across the whole visual field, but declines with increasing retinal eccentricity. Therefore, I investigated how the different temporal response-properties of neurons of the M- and the P-pathways influence self-organization in the visual cortex, and discussed possible consequences for the coding of visual objects at different locations of the visual field. Specifically, I studied the influence of stimulus-motion on the self-organization of lateral connections in a network-model of spiking neurons with Hebbian learning. Low stimulus velocities lead to horizontal connections well adapted to the coding of the spatial structure within the visual input, while higher stimulus velocities lead to connections which subserve the coding of the stimulus movement direction. This suggests that the temporal lowpass properties of P-neurons subserve the coding of spatial stimulus attributes (form) in the visual cortex, while the temporal bandpass properties of M-neurons support the coding of spatio-temporal stimulus attributes (movement direction). Hence, the central representation of the visual field may be well adapted to the encoding of spatial object properties due to the strong contribution of P-neurons. The peripheral representation may be better adapted to the processing of motion

From receptive profiles to a metric model of V1

In this work we show how to construct connectivity kernels induced by the receptive profiles of simple cells of the primary visual cortex (V1). These kernels are directly defined by the shape of such profiles: this provides a metric model for the functional architecture of V1, whose global geometry is determined by the reciprocal interactions between local elements. Our construction adapts to any bank of filters chosen to represent a set of receptive profiles, since it does not require any structure on the parameterization of the family. The connectivity kernel that we define carries a geometrical structure consistent with the well-known properties of long-range horizontal connections in V1, and it is compatible with the perceptual rules synthesized by the concept of association field. These characteristics are still present when the kernel is constructed from a bank of filters arising from an unsupervised learning algorithm.Comment: 25 pages, 18 figures. Added acknowledgement

The Impact of 2-D and 3-D Grouping Cues on Depth From Binocular Disparity

Stereopsis is a powerful source of information about the relative depth of objects in the world. In isolation, humans can see depth from binocular disparity without any other depth cues. However, many different stimulus properties can dramatically influence the depth we perceive. For example, there is an abundance of research showing that the configuration of a stimulus can impact the percept of depth, in some cases diminishing the amount of depth experience. Much of the previous research has focused on discrimination thresholds; in one example, stereoacuity for a pair of vertical lines was shown to be markedly reduced when these lines were connected to form a rectangle apparently slanted in depth (eg: McKee, 1983). The contribution of Gestalt figural grouping to this phenomenon has not been studied. This dissertation addresses the role that perceptual grouping plays in the recovery of suprathreshold depth from disparity. First, I measured the impact of perceptual closure on depth magnitude. Observers estimated the separation in depth of a pair of vertical lines as the amount of perceptual closure was varied. In a series of experiments, I characterized the 2-D and 3-D properties that contribute to 3-D closure and the estimates of apparent depth. Estimates of perceived depth were highly correlated to the strength of subjective closure. Furthermore, I highlighted the perceptual consequences (both costs and benefits) of a new disparity-based grouping cue that interacts with perceived closure, which I call good stereoscopic continuation. This cue was shown to promote detection in a visual search task but reduces depth percepts compared to isolated features. Taken together, the results reported here show that specific 2-D and 3-D grouping constraints are required to promote recovery of a 3-D object. As a consequence, quantitative depth is reduced, but the object is rapidly detected in a visual search task. I propose that these phenomena are the result of object-based disparity smoothing operations that enhance object cohesion

Incremental grouping of image elements in vision

One important task for the visual system is to group image elements that belong to an object and to segregate them from other objects and the background. We here present an incremental grouping theory (IGT) that addresses the role of object-based attention in perceptual grouping at a psychological level and, at the same time, outlines the mechanisms for grouping at the neurophysiological level. The IGT proposes that there are two processes for perceptual grouping. The first process is base grouping and relies on neurons that are tuned to feature conjunctions. Base grouping is fast and occurs in parallel across the visual scene, but not all possible feature conjunctions can be coded as base groupings. If there are no neurons tuned to the relevant feature conjunctions, a second process called incremental grouping comes into play. Incremental grouping is a time-consuming and capacity-limited process that requires the gradual spread of enhanced neuronal activity across the representation of an object in the visual cortex. The spread of enhanced neuronal activity corresponds to the labeling of image elements with object-based attention

Distributed processing and temporal codes in neuronal networks

The cerebral cortex presents itself as a distributed dynamical system with the characteristics of a small world network. The neuronal correlates of cognitive and executive processes often appear to consist of the coordinated activity of large assemblies of widely distributed neurons. These features require mechanisms for the selective routing of signals across densely interconnected networks, the flexible and context dependent binding of neuronal groups into functionally coherent assemblies and the task and attention dependent integration of subsystems. In order to implement these mechanisms, it is proposed that neuronal responses should convey two orthogonal messages in parallel. They should indicate (1) the presence of the feature to which they are tuned and (2) with which other neurons (specific target cells or members of a coherent assembly) they are communicating. The first message is encoded in the discharge frequency of the neurons (rate code) and it is proposed that the second message is contained in the precise timing relationships between individual spikes of distributed neurons (temporal code). It is further proposed that these precise timing relations are established either by the timing of external events (stimulus locking) or by internal timing mechanisms. The latter are assumed to consist of an oscillatory modulation of neuronal responses in different frequency bands that cover a broad frequency range from <2 Hz (delta) to >40 Hz (gamma) and ripples. These oscillations limit the communication of cells to short temporal windows whereby the duration of these windows decreases with oscillation frequency. Thus, by varying the phase relationship between oscillating groups, networks of functionally cooperating neurons can be flexibly configurated within hard wired networks. Moreover, by synchronizing the spikes emitted by neuronal populations, the saliency of their responses can be enhanced due to the coincidence sensitivity of receiving neurons in very much the same way as can be achieved by increasing the discharge rate. Experimental evidence will be reviewed in support of the coexistence of rate and temporal codes. Evidence will also be provided that disturbances of temporal coding mechanisms are likely to be one of the pathophysiological mechanisms in schizophrenia

Bioplausible multiscale filtering in retino-cortical processing as a mechanism in perceptual grouping

Why does our visual system fail to reconstruct reality, when we look at certain patterns? Where do Geometrical illusions start to emerge in the visual pathway? How far should we take computational models of vision with the same visual ability to detect illusions as we do? This study addresses these questions, by focusing on a specific underlying neural mechanism involved in our visual experiences that affects our final perception. Among many types of visual illusion, Geometrical and, in particular, Tilt Illusions are rather important, being characterized by misperception of geometric patterns involving lines and tiles in combination with contrasting orientation, size or position. Over the last decade, many new neurophysiological experiments have led to new insights as to how, when and where retinal processing takes place, and the encoding nature of the retinal representation that is sent to the cortex for further processing. Based on these neurobiological discoveries, we provide computer simulation evidence from modelling retinal ganglion cells responses to some complex Tilt Illusions, suggesting that the emergence of tilt in these illusions is partially related to the interaction of multiscale visual processing performed in the retina. The output of our low-level filtering model is presented for several types of Tilt Illusion, predicting that the final tilt percept arises from multiple-scale processing of the Differences of Gaussians and the perceptual interaction of foreground and background elements. Our results suggest that this model has a high potential in revealing the underlying mechanism connecting low-level filtering approaches to mid- and high-level explanations such as Anchoring theory and Perceptual grouping.Comment: 23 pages, 8 figures, Brain Informatics journal: Full text access: https://link.springer.com/article/10.1007/s40708-017-0072-