Learning long-range spatial dependencies with horizontal gated-recurrent units

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce the horizontal gated-recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures which have orders of magnitude more free parameters. We further discuss the biological plausibility of the hGRU in comparison to anatomical data from the visual cortex as well as human behavioral data on a classic contour detection task.Comment: Published at NeurIPS 2018 https://papers.nips.cc/paper/7300-learning-long-range-spatial-dependencies-with-horizontal-gated-recurrent-unit

A Neural Model of How Horizontal and Interlaminar Connections of Visual Cortex Develop into Adult Circuits that Carry Out Perceptual Grouping and Learning

A neural model suggests how horizontal and interlaminar connections in visual cortical areas V1 and V2 develop within a laminar cortical architecture and give rise to adult visual percepts. The model suggests how mechanisms that control cortical development in the infant lead to properties of adult cortical anatomy, neurophysiology, and visual perception. The model clarifies how excitatory and inhibitory connections can develop stably by maintaining a balance between excitation and inhibition. The growth of long-range excitatory horizontal connections between layer 2/3 pyramidal cells is balanced against that of short-range disynaptie interneuronal connections. The growth of excitatory on-center connections from layer 6-to-1 is balanced against that of inhibitory interneuronal off-surround connections. These balanced connections interact via intracortical and intercortical feedback to realize properties of perceptual grouping, attention, and perceptual learning in the adult, and help to explain the observed variability in the number and temporal distribution of spikes emitted by cortical neurons. The model replicates cortical point spread functions and psychophysical data on the strength of real and illusory contours. The on-center off-surround layer 6-to-4 circuit enables top-down attentional signals from area V2 to modulate, or attentionally prime, layer 4 cells in area VI without fully activating them. This modulatory circuit also enables adult perceptual learning within cortical area, V1 and V2 to proceed in a stable way.Defense Advanced Research Projects Agency and Office of Naval Hesearch (N00014-95-l-0109); National Science Foundation (IRI-97-20333); Office of Naval Research (N00014-95-1-0657

A robust contour detection operator with combined push-pull inhibition and surround suppression

Contour detection is a salient operation in many computer vision applications as it extracts features that are important for distinguishing objects in scenes. It is believed to be a primary role of simple cells in visual cortex of the mammalian brain. Many of such cells receive push-pull inhibition or surround suppression. We propose a computational model that exhibits a combination of these two phenomena. It is based on two existing models, which have been proven to be very effective for contour detection. In particular, we introduce a brain-inspired contour operator that combines push-pull and surround inhibition. It turns out that this combination results in a more effective contour detector, which suppresses texture while keeping the strongest responses to lines and edges, when compared to existing models. The proposed model consists of a Combination of Receptive Field (or CORF) model with push-pull inhibition, extended with surround suppression. We demonstrate the effectiveness of the proposed approach on the RuG and Berkeley benchmark data sets of 40 and 500 images, respectively. The proposed push-pull CORF operator with surround suppression outperforms the one without suppression with high statistical significance

A Push-Pull CORF Model of a Simple Cell with Antiphase Inhibition Improves SNR and Contour Detection

We propose a computational model of a simple cell with push-pull inhibition, a property that is observed in many real simple cells. It is based on an existing model called Combination of Receptive Fields or CORF for brevity. A CORF model uses as afferent inputs the responses of model LGN cells with appropriately aligned center-surround receptive fields, and combines their output with a weighted geometric mean. The output of the proposed model simple cell with push-pull inhibition, which we call push-pull CORF, is computed as the response of a CORF model cell that is selective for a stimulus with preferred orientation and preferred contrast minus a fraction of the response of a CORF model cell that responds to the same stimulus but of opposite contrast. We demonstrate that the proposed push-pull CORF model improves signal-to-noise ratio (SNR) and achieves further properties that are observed in real simple cells, namely separability of spatial frequency and orientation as well as contrast-dependent changes in spatial frequency tuning. We also demonstrate the effectiveness of the proposed push-pull CORF model in contour detection, which is believed to be the primary biological role of simple cells. We use the RuG (40 images) and Berkeley (500 images) benchmark data sets of images with natural scenes and show that the proposed model outperforms, with very high statistical significance, the basic CORF model without inhibition, Gabor-based models with isotropic surround inhibition, and the Canny edge detector. The push-pull CORF model that we propose is a contribution to a better understanding of how visual information is processed in the brain as it provides the ability to reproduce a wider range of properties exhibited by real simple cells. As a result of push-pull inhibition a CORF model exhibits an improved SNR, which is the reason for a more effective contour detection.</p

A Neural Model of How Horizontal and Interlaminar Connections of Visual Cortex Develop into Adult Circuits that Carry Out Perceptual Grouping and Learning

A neural model suggests how horizontal and interlaminar connections in visual cortical areas Vl and V2 develop within a laminar cortical architecture and give rise to adult visual percepts. The model suggests how mechanisms that control cortical development in the infant lead to properties of adult cortical anatomy, neurophysiology, and visual perception. The model clarifies how excitatory and inhibitory connections can develop stably by maintaining a balance between excitation and inhibition. The growth of long-range excitatory horizontal connections between layer 2/3 pyramidal cells is balanced against that of short-range disynaptic interneuronal connections. The growth of excitatory on-center connections from layer 6-to-4 is balanced against that of inhibitory interneuronal off-surround connections. These balanced connections interact via intracortical and intercortical feedback to realize properties of perceptual grouping, attention, and perceptual learning in the adult, and help to explain the observed variability in the number and temporal distribution of spikes emitted by cortical neurons. The model replicates cortical point spread functions and psychophysical data on the strength of real and illusory contours. The on-center off-surround layer 6-to-4 circuit enables top-clown attentional signals from area V2 to modulate, or attentionally prime, layer 4 cells in area Vl without fully activating them. This modulatory circuit also enables adult perceptual learning within cortical area Vl and V2 to proceed in a stable way.Defense Advanced Research Projects Agency and the Office of Naval Research (N00014-95-1-0409); National Science Foundation (IRI-97-20333); Office of Naval Research (N00014-95-1-0657

The Role of Early Recurrence in Improving Visual Representations

This dissertation proposes a computational model of early vision with recurrence, termed as early recurrence. The idea is motivated from the research of the primate vision. Specifically, the proposed model relies on the following four observations. 1) The primate visual system includes two main visual pathways: the dorsal pathway and the ventral pathway; 2) The two pathways respond to different visual features; 3) The neurons of the dorsal pathway conduct visual information faster than that of the neurons of the ventral pathway; 4) There are lower-level feedback connections from the dorsal pathway to the ventral pathway. As such, the primate visual system may implement a recurrent mechanism to improve visual representations of the ventral pathway. Our work starts from a comprehensive review of the literature, based on which a conceptualization of early recurrence is proposed. Early recurrence manifests itself as a form of surround suppression. We propose that early recurrence is capable of refining the ventral processing using results of the dorsal processing. Our work further defines a set of computational components to formalize early recurrence. Although we do not intend to model the true nature of biology, to verify that the proposed computation is biologically consistent, we have applied the model to simulate a neurophysiological experiment of a bar-and-checkerboard and a psychological experiment involving a moving contour illusion. Simulation results indicated that the proposed computation behaviourally reproduces the original observations. The ultimate goal of this work is to investigate whether the proposal is capable of improving computer vision applications. To do this, we have applied the model to a variety of applications, including visual saliency and contour detection. Based on comparisons against the state-of-the-art, we conclude that the proposed model of early recurrence sheds light on a generally applicable yet lightweight approach to boost real-life application performance