Sparse visual models for biologically inspired sensorimotor control

Given the importance of using resources efficiently in the competition for survival, it is reasonable to think that natural evolution has discovered efficient cortical coding strategies for representing natural visual information. Sparse representations have intrinsic advantages in terms of fault-tolerance and low-power consumption potential, and can therefore be attractive for robot sensorimotor control with powerful dispositions for decision-making. Inspired by the mammalian brain and its visual ventral pathway, we present in this paper a hierarchical sparse coding network architecture that extracts visual features for use in sensorimotor control. Testing with natural images demonstrates that this sparse coding facilitates processing and learning in subsequent layers. Previous studies have shown how the responses of complex cells could be sparsely represented by a higher-order neural layer. Here we extend sparse coding in each network layer, showing that detailed modeling of earlier stages in the visual pathway enhances the characteristics of the receptive fields developed in subsequent stages. The yield network is more dynamic with richer and more biologically plausible input and output representation

Contour integration: Psychophysical, neurophysiological, and computational perspectives

One of the important roles of our visual system is to detect and segregate objects. Neurons in the early visual system extract local image features from the visual scene. To combine these features into separate, global objects, the visual system must perform some kind of grouping operation. One such operation is contour integration. Contours form the outlines of objects, and are the first step in shape perception. We discuss the mechanism of contour integration from psychophysical, neurophysiological, and computational perspectives

Contour extracting networks in early extrastriate cortex

Neurons in the visual cortex process a local region of visual space, but in order to adequately analyze natural images, neurons need to interact. The notion of an ?association field? proposes that neurons interact to extract extended contours. Here, we identify the site and properties of contour integration mechanisms. We used functional magnetic resonance imaging (fMRI) and population receptive field (pRF) analyses. We devised pRF mapping stimuli consisting of contours. We isolated the contribution of contour integration mechanisms to the pRF by manipulating the contour content. This stimulus manipulation led to systematic changes in pRF size. Whereas a bank of Gabor filters quantitatively explains pRF size changes in V1, only V2/V3 pRF sizes match the predictions of the association field. pRF size changes in later visual field maps, hV4, LO-1, and LO-2 do not follow either prediction and are probably driven by distinct classical receptive field properties or other extraclassical integration mechanisms. These pRF changes do not follow conventional fMRI signal strength measures. Therefore, analyses of pRF changes provide a novel computational neuroimaging approach to investigating neural interactions. We interpreted these results as evidence for neural interactions along co-oriented, cocircular receptive fields in the early extrastriate visual cortex (V2/V3), consistent with the notion of a contour association field

Neural responses to dynamic adaptation reveal the dissociation between the processing of the shape of contours and textures

This research was supported by a Leverhulme Trust grant (RPG-2016-056) awarded to Elena Gheorghiu (PI) and Jasna Martinovic (co-PI). The C code used to generate the contours, written in conjunction with routines from the VISAGE graphics library (Cambridge Research System) was modified from code originally written by Frederick A. A. Kingdom. We would like to thank Frederick Kingdom for helping with the development of the original C code.Peer reviewedPostprin

BOLD responses in human V1 to local structure in natural scenes: Implications for theories of visual coding

In this study we tested predictions of two important theories of visual coding, contrast energy and sparse coding theory, on the dependence of population activity level and metabolic demands on spatial structure of the visual input. With carefully calibrated displays we find that in humans neither the V1 blood oxygenation level dependent (BOLD) response nor the initial visually evoked fields in magnetoencephalography (MEG) are sensitive to phase perturbations in photographs of natural scenes. As a control, we quantitatively show that the applied phase perturbations decrease sparseness (kurtosis) of our stimuli but preserve their root mean square (RMS) contrast. Importantly, we show that the lack of sensitivity of the V1 population response level to phase perturbations is not due to a lack of sensitivity of our methods because V1 responses were highly sensitive to variations of image RMS contrast. Our results suggest that the transition from a sparse to a distributed neural code in the early visual system induced by reducing image sparseness has negligible consequences for population metabolic cost. This result imposes a novel and important empirical constraint on quantitative models of sparse coding: Population metabolic rate and population activation level is sensitive to second order statistics (RMS contrast) of the input but not to its spatial phase and fourth order statistics (kurtosis)

Statistics of gradient directions in natural images.

Interest in finding statistical regularities in natural images has been growing since the advent of information theory and the advancement of the efficient coding hypothesis that the human visual system is optimised to encode natural visual stimuli. In this thesis, a statistical analysis of gradient directions in an ensemble of natural images is reported. Information-theoretic measures have been used to compute the amount of dependency which exists between triples of gradient directions at separate image locations. Control experiments are performed on other image classes: phase randomized natural images, whitened natural images, and Gaussian noise images. The main results show that for an ensemble of natural images the average amount of de pendency between two and three gradient directions is the same as for an ensemble of phase randomized natural images. This result does not extend to i) the amount dependency between gradient magnitudes, ii) gradient directions at high gradient magnitude locations, or iii) individual natural images. Furthermore, no significant synergetic dependencies are found between triples of gradient directions in an ensemble natural images a synergetic dependency is an increase in dependency between a pair of gradient directions given the interaction of a third gradient direction. Additional experiments are performed to establish both the generality and specificity of the main results by studying the gradient direction dependencies of ensembles of noise (random phases) images with varying power law power spectra. The results of the additional experiments indicate that, for ensembles of images with varying power law power spectra, the amount of dependency between two and three gradient directions is determined by the ensemble's mean power spectrum rather than the phase spectrum. A framework is also presented for future work and preliminary results are provided for the dependency between second order derivative measurements (shape index) for up to 9-point configurations

The 'aperture problem' in complex moving scenes

The initial encoding of direction by mammals occurs in striate cortex by neurons with small receptive fields that are tuned to narrow bands of the spatiotemporal frequency spectrum. Individual neurons are unable to signal the global direction of 2D motion and are instead sensitive to the 1D component of motion perpendicular to a moving edge. To compute 2D velocity, it is necessary to integrate over a range of 1D velocity sensors. In this work I probe the ability of the visual system to compute 2D velocity from a range of stimulus classes, including naturally contoured scenes, natural scenes and a global-Gabor array. My research shows that the motion stream is highly sensitive to the distribution of local orientations present in a moving image, but is largely insensitive to their spatial second-order statistics. I present a computational model of two-dimensional motion processing that is able to derive precise estimates of 2D motion directly from complex natural scenes. The model produces errors when confronted with stimuli composed of anisotropic orientation configurations and is able to capture many of the biases and errors experienced by human observers. Finally, I argue that observers’ misperceptions of 2D motion does not reflect a sub-optimal 2D motion strategy, but reflects a compromise between the competing requirements of defining motions in a spatially discrete manner across space, and the ability to accurately estimate 1D motions, on which the computation of 2D velocity must rely

The What and Why of Binding: The Modeler's Perspective

In attempts to formulate a computational understanding of brain function, one of the fundamental concerns is the data structure by which the brain represents information. For many decades, a conceptual framework has dominated the thinking of both brain modelers and neurobiologists. That framework is referred to here as "classical neural networks." It is well supported by experimental data, although it may be incomplete. A characterization of this framework will be offered in the next section. Difficulties in modeling important functional aspects of the brain on the basis of classical neural networks alone have led to the recognition that another, general mechanism must be invoked to explain brain function. That mechanism I call "binding." Binding by neural signal synchrony had been mentioned several times in the liter ature (Lege´ndy, 1970; Milner, 1974) before it was fully formulated as a general phenomenon (von der Malsburg, 1981). Although experimental evidence for neural syn chrony was soon found, the idea was largely ignored for many years. Only recently has it become a topic of animated discussion. In what follows, I will summarize the nature and the roots of the idea of binding, especially of temporal binding, and will discuss some of the objec tions raised against it

Visual Cortex

The neurosciences have experienced tremendous and wonderful progress in many areas, and the spectrum encompassing the neurosciences is expansive. Suffice it to mention a few classical fields: electrophysiology, genetics, physics, computer sciences, and more recently, social and marketing neurosciences. Of course, this large growth resulted in the production of many books. Perhaps the visual system and the visual cortex were in the vanguard because most animals do not produce their own light and offer thus the invaluable advantage of allowing investigators to conduct experiments in full control of the stimulus. In addition, the fascinating evolution of scientific techniques, the immense productivity of recent research, and the ensuing literature make it virtually impossible to publish in a single volume all worthwhile work accomplished throughout the scientific world. The days when a single individual, as Diderot, could undertake the production of an encyclopedia are gone forever. Indeed most approaches to studying the nervous system are valid and neuroscientists produce an almost astronomical number of interesting data accompanied by extremely worthy hypotheses which in turn generate new ventures in search of brain functions. Yet, it is fully justified to make an encore and to publish a book dedicated to visual cortex and beyond. Many reasons validate a book assembling chapters written by active researchers. Each has the opportunity to bind together data and explore original ideas whose fate will not fall into the hands of uncompromising reviewers of traditional journals. This book focuses on the cerebral cortex with a large emphasis on vision. Yet it offers the reader diverse approaches employed to investigate the brain, for instance, computer simulation, cellular responses, or rivalry between various targets and goal directed actions. This volume thus covers a large spectrum of research even though it is impossible to include all topics in the extremely diverse field of neurosciences