Learning long-range spatial dependencies with horizontal gated-recurrent units

Progress in deep learning has spawned great successes in many engineering applications. As a prime example, convolutional neural networks, a type of feedforward neural networks, are now approaching -- and sometimes even surpassing -- human accuracy on a variety of visual recognition tasks. Here, however, we show that these neural networks and their recent extensions struggle in recognition tasks where co-dependent visual features must be detected over long spatial ranges. We introduce the horizontal gated-recurrent unit (hGRU) to learn intrinsic horizontal connections -- both within and across feature columns. We demonstrate that a single hGRU layer matches or outperforms all tested feedforward hierarchical baselines including state-of-the-art architectures which have orders of magnitude more free parameters. We further discuss the biological plausibility of the hGRU in comparison to anatomical data from the visual cortex as well as human behavioral data on a classic contour detection task.Comment: Published at NeurIPS 2018 https://papers.nips.cc/paper/7300-learning-long-range-spatial-dependencies-with-horizontal-gated-recurrent-unit

A new method for indirect mass measurements using the integral charge asymmetry at the LHC

International audienceProcesses producing a charged final state at the LHC most often have a positive or null integral charge asymmetry. We propose a novel method for an indirect measurement of the mass of these final states based upon the process integral charge asymmetry. We present this method in three stages. Firstly, the theoretical prediction of the integral charge asymmetry and its related uncertainties are studied through parton level cross sections calculations. Secondly, the experimental extraction of the integral charge asymmetry of a given signal, in the presence of some background, is performed using particle level simulations. Process dependent templates enable to convert the measured integral charge asymmetry into an estimated mass of the charged final state. Thirdly, a combination of the experimental and the theoretical uncertainties determines the full uncertainty of the indirect mass measurement. This new method applies to all charged current processes at the LHC. In this article, we demonstrate its effectiveness at extracting the mass of the W boson, as a first step, and the sum of the masses of a chargino and a neutralino in case these supersymmetric particles are produced by pair, as a second step

Fixing the problems of deep neural networks will require better training data and learning algorithms

Bowers and colleagues argue that DNNs are poor models of biological vision because they often learn to rival human accuracy by relying on strategies that differ markedly from those of humans. We show that this problem is worsening as DNNs are becoming larger-scale and increasingly more accurate, and prescribe methods for building DNNs that can reliably model biological vision.Comment: Published as a commentary in Behavioral and Brain Science

Towards a Unified Computational Model of Contextual Interactions across Visual Modalities

The perception of a stimulus is largely determined by its surrounding. Examples abound from color (Land and McCann, 1971), disparity (Westheimer, 1986) and motion induction (Anstis and Casco, 2006) to orientation tilt effects (O’Toole and Wenderoth, 1976). Some of these phenomena have been studied individually using monkey neurophysiology techniques. In these experiments, a center stimulus is typically used to probe a cell’s classical “center” receptive field (cRF), whose activity is then modulated by an annular “surround” (extra-cRF) stimulus. While this center-surround integration (CSI) has been well characterized, a theoretical framework which unifies these different phenomena across visual modalities is lacking. Here, we present an extension of a popular cortical inhibition circuit, divisive normalization (Carandini and Heeger, 2011), which yields a computational model that is consistent with experimental data across visual modalities. We have found that a common characteristic of CSI across modalities is a shift in neural population responses induced by surround activity. Typical implementations of the divisive normalization model rely on gain control mechanisms from an ‘untuned’ suppressive pool of cells; that is, the identity of that pool is the same for every cell being suppressed. As such, the circuit cannot account for the selective shift in population response curves observed in contextual effects. In contrast, we show that the addition of an extra-classical suppressive ‘tuned’ pool of cells which selectively inhibits different parts of a population response curve is sufficient to explain complex shifts in population tuning responses. Overall, our results suggest that a normalization circuit based on two forms of inhibition, gain control and selective suppression, captures shifts in population responses associated with CSI and yields a model that seems consistent with contextual phenomena across visual modalities