Emergence of Visual Saliency from Natural Scenes via Context-Mediated Probability Distributions Coding

Visual saliency is the perceptual quality that makes some items in visual scenes stand out from their immediate contexts. Visual saliency plays important roles in natural vision in that saliency can direct eye movements, deploy attention, and facilitate tasks like object detection and scene understanding. A central unsolved issue is: What features should be encoded in the early visual cortex for detecting salient features in natural scenes? To explore this important issue, we propose a hypothesis that visual saliency is based on efficient encoding of the probability distributions (PDs) of visual variables in specific contexts in natural scenes, referred to as context-mediated PDs in natural scenes. In this concept, computational units in the model of the early visual system do not act as feature detectors but rather as estimators of the context-mediated PDs of a full range of visual variables in natural scenes, which directly give rise to a measure of visual saliency of any input stimulus. To test this hypothesis, we developed a model of the context-mediated PDs in natural scenes using a modified algorithm for independent component analysis (ICA) and derived a measure of visual saliency based on these PDs estimated from a set of natural scenes. We demonstrated that visual saliency based on the context-mediated PDs in natural scenes effectively predicts human gaze in free-viewing of both static and dynamic natural scenes. This study suggests that the computation based on the context-mediated PDs of visual variables in natural scenes may underlie the neural mechanism in the early visual cortex for detecting salient features in natural scenes

Low level constraints on dynamic contour path integration

Contour integration is a fundamental visual process. The constraints on integrating discrete contour elements and the associated neural mechanisms have typically been investigated using static contour paths. However, in our dynamic natural environment objects and scenes vary over space and time. With the aim of investigating the parameters affecting spatiotemporal contour path integration, we measured human contrast detection performance of a briefly presented foveal target embedded in dynamic collinear stimulus sequences (comprising five short 'predictor' bars appearing consecutively towards the fovea, followed by the 'target' bar) in four experiments. The data showed that participants' target detection performance was relatively unchanged when individual contour elements were separated by up to 2° spatial gap or 200ms temporal gap. Randomising the luminance contrast or colour of the predictors, on the other hand, had similar detrimental effect on grouping dynamic contour path and subsequent target detection performance. Randomising the orientation of the predictors reduced target detection performance greater than introducing misalignment relative to the contour path. The results suggest that the visual system integrates dynamic path elements to bias target detection even when the continuity of path is disrupted in terms of spatial (2°), temporal (200ms), colour (over 10 colours) and luminance (-25% to 25%) information. We discuss how the findings can be largely reconciled within the functioning of V1 horizontal connections

Cortical simple cells can extract achromatic information from the multiplexed chromatic and achromatic signals in the parvocellular pathway

AbstractP cells, which carry both achromatic and chromatic information, are largely responsible for achromatic acuity and contrast sensitivity. The P cell achromatic information must be separated from the chromatic information to be useful. Cortical simple cells are well suited to the extraction of achromatic information by spatial bandpass filtering. Bandpass filtering of Type I P cells by cortical simple cells yields an achromatic signal with a residual chromatic response. The bandpass model makes predictions in accord with existing physiological data and explains the role of a heretofore puzzling class of cortical cells, which have bandpass tuning for both achromatic and chromatic modulations. The model is shown to be related to a previously postulated class of ideal detectors. Finally, the model is used to make a number of physiological and psychophysical predictions

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Creating effective focus cues in multi-plane 3D displays.

Focus cues are incorrect in conventional stereoscopic displays. This causes a dissociation of vergence and accommodation, which leads to visual fatigue and perceptual distortions. Multi-plane displays can minimize these problems by creating nearly correct focus cues. But to create the appearance of continuous depth in a multi-plane display, one needs to use depth-weighted blending: i.e., distribute light intensity between adjacent planes. Akeley et al. [ACM Trans. Graph. 23, 804 (2004)] and Liu and Hua [Opt. Express 18, 11562 (2009)] described rather different rules for depth-weighted blending. We examined the effectiveness of those and other rules using a model of a typical human eye and biologically plausible metrics for image quality. We find that the linear blending rule proposed by Akeley and colleagues [ACM Trans. Graph. 23, 804 (2004)] is the best solution for natural stimuli

Processing of natural temporal stimuli by macaque retinal ganglion cells

This study quantifies the performance of primate retinal ganglion cells in response to natural stimuli. Stimuli were confined to the temporal and chromatic domains and were derived from two contrasting environments, one typically northern European and the other a flower show. The performance of the cells was evaluated by investigating variability of cell responses to repeated stimulus presentations and by comparing measured to model responses. Both analyses yielded a quantity called the coherence rate (in bits per second), which is related to the information rate. Magnocellular (MC) cells yielded coherence rates of up to 100 bits/sec, rates of parvocellular (PC) cells were much lower, and short wavelength (S)-cone-driven ganglion cells yielded intermediate rates. The modeling approach showed that for MC cells, coherence rates were generated almost exclusively by the luminance content of the stimulus. Coherence rates of PC cells were also dominated by achromatic content. This is a consequence of the stimulus structure; luminance varied much more in the natural environment than chromaticity. Only approximately one-sixth of the coherence rate of the PC cells derived from chromatic content, and it was dominated by frequencies below 10 Hz. S-cone-driven ganglion cells also yielded coherence rates dominated by low frequencies. Below 2–3 Hz, PC cell signals contained more power than those of MC cells. Response variation between individual ganglion cells of a particular class was analyzed by constructing generic cells, the properties of which may be relevant for performance higher in the visual system. The approach used here helps define retinal modules useful for studies of higher visual processing of natural stimuli