Natural Image Coding in V1: How Much Use is Orientation Selectivity?

Orientation selectivity is the most striking feature of simple cell coding in V1 which has been shown to emerge from the reduction of higher-order correlations in natural images in a large variety of statistical image models. The most parsimonious one among these models is linear Independent Component Analysis (ICA), whereas second-order decorrelation transformations such as Principal Component Analysis (PCA) do not yield oriented filters. Because of this finding it has been suggested that the emergence of orientation selectivity may be explained by higher-order redundancy reduction. In order to assess the tenability of this hypothesis, it is an important empirical question how much more redundancies can be removed with ICA in comparison to PCA, or other second-order decorrelation methods. This question has not yet been settled, as over the last ten years contradicting results have been reported ranging from less than five to more than hundred percent extra gain for ICA. Here, we aim at resolving this conflict by presenting a very careful and comprehensive analysis using three evaluation criteria related to redundancy reduction: In addition to the multi-information and the average log-loss we compute, for the first time, complete rate-distortion curves for ICA in comparison with PCA. Without exception, we find that the advantage of the ICA filters is surprisingly small. Furthermore, we show that a simple spherically symmetric distribution with only two parameters can fit the data even better than the probabilistic model underlying ICA. Since spherically symmetric models are agnostic with respect to the specific filter shapes, we conlude that orientation selectivity is unlikely to play a critical role for redundancy reduction

On the relationship between optical variability, visual saliency, and eye fixations: a computational approach

A hierarchical definition of optical variability is proposed that links physical magnitudes to visual saliency and yields a more reductionist interpretation than previous approaches. This definition is shown to be grounded on the classical efficient coding hypothesis. Moreover, we propose that a major goal of contextual adaptation mechanisms is to ensure the invariance of the behavior that the contribution of an image point to optical variability elicits in the visual system. This hypothesis and the necessary assumptions are tested through the comparison with human fixations and state-of-the-art approaches to saliency in three open access eye-tracking datasets, including one devoted to images with faces, as well as in a novel experiment using hyperspectral representations of surface reflectance. The results on faces yield a significant reduction of the potential strength of semantic influences compared to previous works. The results on hyperspectral images support the assumptions to estimate optical variability. As well, the proposed approach explains quantitative results related to a visual illusion observed for images of corners, which does not involve eye movementsS

Target detection using saliency-based attention

Most models of visual search, whether involving overt eye movements or covert shifts of attention, are based on the concept of a "saliency map", that is, an explicit two-dimensional map that encodes the saliency or conspicuity of objects in the visual environment. Competition among neurons in this map gives rise to a single winning location that corresponds to the next attended target. Inhibiting this location automatically allows the system to attend to the next most salient location. We describe a detailed computer implementation of such a scheme, focusing on the problem of combining information across modalities, here orientation, intensity and color information, in a purely stimulus-driven manner. We have successfully applied this model to a wide range of target detection tasks, using synthetic and natural stimuli. Performance has however remained difficult to objectively evaluate on natural scenes, because no objective reference was available for comparison. We here present predicted search times for our model on the Search2 database of rural scenes containing a military vehicle. Overall, we found a poor correlation between human and model search times. Further analysis however revealed that in 3/4 of the images, the model appeared to detect the target faster than humans (for comparison, we calibrated the model’s arbitrary internal time frame such that no more than 2-4 image locations were visited per second). It hence seems that this model, which had originally been designed not to find small, hidden military vehicles, but rather to find the few most obviously conspicuous objects in an image, performed as an efficient target detector on the Search2 dataset

Chromatic processing in the zebrafish (Danio rerio) inner retina: bipolar cell physiology and open hardware designs for spectrally accurate stimulation under two-photon

Colour vision describes the ability of animals to differentiate objects based on their spectral reflectance properties independent of light intensity. It is an essential evolutionary trait that allows species to efficiently forage for food, avoid predation, break camouflage, communicate with conspecifics, or to find mates. Zebrafish is a powerful model for studying colour vision as it possesses four cone-photoreceptor types which can be categorised as Red-, Green-, Blue- and UV-sensitive. From first principles, its retina therefore holds the potential to process diverse chromatic computations. In the presented work, the focus was on retinal bipolar cells (BC). These are the retina’s first projection neurons. They receive inputs from the photoreceptors in the outer retina, and send their axon terminals to the inner retina, the inner plexiform layer (IPL). Diverse types within this class of interneuron shape light responses collected by the photoreceptor array into parallel channels with diverse spectral properties. BCs also make connections with all other neuron types within the retina, including horizontal cells in the outer retina, and amacrine as well as retinal ganglion cells in the inner retina. This makes them a central hub for spectral processing within the retina. By combining genetically encoded calcium indicator and two-photon microscopy, light-driven activity from larval zebrafish BC synaptic terminals was systematically recorded in vivo. Synaptic responses to tetrachromatic light stimulation unveiled an unprecedented degree of visual specialisation, including retinal regions dedicated to distinct light-guided behaviours. These regional characteristics were further correlated to functional BC types which were strongly associated with specific retinal positions and axonal stratification depths. Overall, BC projections to the inner plexiform layer displayed a sophisticated level of organisation, structured into chromatic and achromatic functional layers which systematically adjusted their response profiles across the eye to match natural spectral input statistics. Together, these findings bolster our understanding of “colour-processing” in this animal’s inner retina and suggest that unlike in mammals, teleost fish BCs already encode complex chromatic responses in the inner plexiform layer before driving retinal ganglion cells. Additionally, the study of colour vision from an organism requires precise control over the light stimuli’s temporal, spatial and spectral features. Therefore, chromatic stimulators, designed to be combined with two-photon microscopy, were developed throughout this work. These devices allowed circumventing experimental limitations, such as spectral crosstalk between the microscope and the stimulus light. Furthermore, they were conceived as open source projects to be easily replicated and adapted to any organism’s retina with different spectral sensitivities through the free control over the number and spectra of stimulation light sources. These open source projects originated from the desire to set up a stimulation standard for the field of visual neuroscience

A saliency-based search mechanism for overt and covert shifts of visual attention

Most models of visual search, whether involving overt eye movements or covert shifts of attention, are based on the concept of a saliency map, that is, an explicit two-dimensional map that encodes the saliency or conspicuity of objects in the visual environment. Competition among neurons in this map gives rise to a single winning location that corresponds to the next attended target. Inhibiting this location automatically allows the system to attend to the next most salient location. We describe a detailed computer implementation of such a scheme, focusing on the problem of combining information across modalities, here orientation, intensity and color information, in a purely stimulus-driven manner. The model is applied to common psychophysical stimuli as well as to a very demanding visual search task. Its successful performance is used to address the extent to which the primate visual system carries out visual search via one or more such saliency maps and how this can be tested