Flexible Gating of Contextual Influences in Natural Vision

An appealing hypothesis suggests that neurons represent inputs in a coordinate system that is matched to the statistical structure of images in the natural environment. I discuss theoretical work on unsupervised learning of statistical regularities in natural images. In the model, Bayesian inference amounts to a generalized form of divisive normalization, a canonical computation that has been implicated in many neural areas. In our framework, divisive normalization is flexible: it is recruited only when the image is inferred to contain dependencies, and muted otherwise. I particularly focus on recent work in which we have applied this approach to understanding spatial context effects in visual cortical processing of natural inputs

Integrating Flexible Normalization into Mid-Level Representations of Deep Convolutional Neural Networks

Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to mid-level representations of deep CNNs as a tractable way to study contextual normalization mechanisms in mid-level cortical areas. This approach captures non-trivial spatial dependencies among mid-level features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high order features, geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in mid-level cortical areas. We also expect this approach to be useful as part of the CNN toolkit, therefore going beyond more restrictive fixed forms of normalization

Cortical Surround Interactions and Perceptual Salience via Natural Scene Statistics

Spatial context in images induces perceptual phenomena associated with salience and modulates the responses of neurons in primary visual cortex (V1). However, the computational and ecological principles underlying contextual effects are incompletely understood. We introduce a model of natural images that includes grouping and segmentation of neighboring features based on their joint statistics, and we interpret the firing rates of V1 neurons as performing optimal recognition in this model. We show that this leads to a substantial generalization of divisive normalization, a computation that is ubiquitous in many neural areas and systems. A main novelty in our model is that the influence of the context on a target stimulus is determined by their degree of statistical dependence. We optimized the parameters of the model on natural image patches, and then simulated neural and perceptual responses on stimuli used in classical experiments. The model reproduces some rich and complex response patterns observed in V1, such as the contrast dependence, orientation tuning and spatial asymmetry of surround suppression, while also allowing for surround facilitation under conditions of weak stimulation. It also mimics the perceptual salience produced by simple displays, and leads to readily testable predictions. Our results provide a principled account of orientation-based contextual modulation in early vision and its sensitivity to the homogeneity and spatial arrangement of inputs, and lends statistical support to the theory that V1 computes visual salience

Natural Sound Statistics and Divisive Normalization in the Auditory System

We explore the statistical properties of natural sound stimuli preprocessed with a bank of linear filters. The responses of such filters exhibit a striking form of statistical dependency, in which the response variance of each filter grows with the response amplitude of filters tuned for nearby frequencies. These dependencies may be substantially reduced using an operation known as divisive normalization, in which the response of each filter is divided by a weighted sum of the rectified responses of other filters. The weights may be chosen to maximize the independence of the normalized responses for an ensemble of natural sounds. We demonstrate that the resulting model accounts for non-linearities in the response characteristics of the auditory nerve, by comparing model simulations to electrophysiological recordings. In previous work (NIPS, 1998) we demonstrated that an analogous model derived from the statistics of natural images accounts for non-linear properties of neur..

A saliency-based bottom-up visual attention model for dynamicscenes analysis

This work proposes a model of visual bottom-up attention for dynamic scene analysis. Our work addsmotion saliency calculations to a neural network model withrealistic temporal dynamics [(e.g., building motion salienceon top of De Brecht and Saiki Neural Networks 19:1467–1474, (2006)]. The resulting network elicits strong transientresponses to moving objects and reaches stability withina biologically plausible time interval. The responses arestatistically different comparing between earlier and latermotion neural activity; and between moving and non-movingobjects. We demonstrate the network on a number of syn-thetic and real dynamical movie examples. We show thatthe model captures the motion saliency asymmetry phenom-enon. In addition, the motion salience computation enablessudden-onset moving objects that are less salient in the staticscene to rise above others. Finally, we include strong consid-eration for the neural latencies, the Lyapunov stability, andthe neural properties being reproduced by the mode