Objects predict fixations better than early saliency

Humans move their eyes while looking at scenes and pictures. Eye movements correlate with shifts in attention and are thought to be a consequence of optimal resource allocation for high-level tasks such as visual recognition. Models of attention, such as “saliency maps,” are often built on the assumption that “early” features (color, contrast, orientation, motion, and so forth) drive attention directly. We explore an alternative hypothesis: Observers attend to “interesting” objects. To test this hypothesis, we measure the eye position of human observers while they inspect photographs of common natural scenes. Our observers perform different tasks: artistic evaluation, analysis of content, and search. Immediately after each presentation, our observers are asked to name objects they saw. Weighted with recall frequency, these objects predict fixations in individual images better than early saliency, irrespective of task. Also, saliency combined with object positions predicts which objects are frequently named. This suggests that early saliency has only an indirect effect on attention, acting through recognized objects. Consequently, rather than treating attention as mere preprocessing step for object recognition, models of both need to be integrated

Characterization of natural illuminants in forests and the use of digital video data to reconstruct illuminant spectra

[[abstract]]We describe illumination spectra in forests and show that they can be accurately recovered from recorded digital video images. Natural illuminant spectra of 238 samples measured in temperate forests were characterized by principal-component analysis. The spectra can be accurately approximated by the mean and the first two principal components. Compared with illumination under open skies, the loci of forest illuminants are displaced toward the green region in the chromaticity plots, and unlike open sky illumination they cannot be characterized by correlated color temperature. We show that it is possible to recover illuminant spectra accurately from digital video images by a linear least-squares-fit estimation technique. The use of digital video data in spectral analysis provides a promising new approach to the studies of the spatial and temporal variation of illumination in natural scenes and the understanding of color vision in natural environments.[[fileno]]2050130010006[[department]]生科

A Large Image Database for Color Constancy Research

We present a study on various statistics relevant to research on color constancy. Many of these analyses could not have been done before simply because a large database for color constancy was not available. Our image database consists of approximately 11,000 images in which the RGB color of the ambient illuminant in each scene is measured. To build such a large database we used a novel set-up consisting of a digital video camera with a neutral gray sphere attached to the camera so that the sphere always appears in the field of view. Using a gray sphere instead of the standard gray card facilitates measurement of the variation in illumination as a function of incident angle. The study focuses on the analysis of the distribution of various illuminants in the natural scenes and the correlation between the rg-chromaticity of colors recorded by the camera and the rg-chromaticity of the ambient illuminant. We also investigate the possibility of improving the performance of the naïve Gray World algorithm by considering a sequence of consecutive frames instead of a single image. The set of images is publicly available and can also be used as a database for testing color constancy algorithms

Color improves edge classification in human vision

Data Availability Statement: All relevant data are within the manuscript and its Supporting Information files.© 2019 Breuil et al. Despite the complexity of the visual world, humans rarely confuse variations in illumination, for example shadows, from variations in material properties, such as paint or stain. This ability to distinguish illumination from material edges is crucial for determining the spatial layout of objects and surfaces in natural scenes. In this study, we explore the role that color (chromatic) cues play in edge classification. We conducted a psychophysical experiment that required subjects to classify edges into illumination and material, in patches taken from images of natural scenes that either contained or did not contain color information. The edge images were of various sizes and were pre-classified into illumination and material, based on inspection of the edge in the context of the whole image from which the edge was extracted. Edge classification performance was found to be superior for the color compared to grayscale images, in keeping with color acting as a cue for edge classification. We defined machine observers sensitive to simple image properties and found that they too classified the edges better with color information, although they failed to capture the effect of image size observed in the psychophysical experiment. Our findings are consistent with previous work suggesting that color information facilitates the identification of material properties, transparency, shadows and the perception of shape-from-shading.IDEX; Canadian Institute of Health. The study was supported by a travel grant from IDEX given to CB and a Canadian Institute of Health Research grant #MOP 123349 given to FK. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript

Time-lapse ratios of cone excitations in natural scenes

The illumination in natural environments varies through the day. Stable inferences about surface color might be supported by spatial ratios of cone excitations from the reflected light, but their invariance has been quantified only for global changes in illuminant spectrum. The aim here was to test their invariance under natural changes in both illumination spectrum and geometry, especially in the distribution of shadows. Time-lapse hyperspectral radiance images were acquired from five outdoor vegetated and nonvegetated scenes. From each scene, 10,000 pairs of points were sampled randomly and ratios measured across time. Mean relative deviations in ratios were generally large, but when sampling was limited to short distances or moderate time intervals, they fell below the level for detecting violations in ratio invariance. When illumination changes with uneven geometry were excluded, they fell further, to levels obtained with global changes in illuminant spectrum alone. Within sampling constraints, ratios of cone excitations, and also of opponent-color combinations, provide an approximately invariant signal for stable surface-color inferences, despite spectral and geometric variations in scene illumination.This work was supported by the Engineering and Physical Sciences Research Council, United Kingdom (Grant Nos. GR/R39412/01, EP/B000257/1, and EP/E056512/1). We thank Iván Marín-Franch for advice on statistical analysis and Oscar González for critical comments on the manuscript

A New Approach for Text String Detection from Natural Scenes By Grouping & Partition

In this paper we have reviewed and analyzed different methods to find strings of characters from natural scene images. We have reviewed different techniques like extraction of character string regions from scenery images based on contours and thickness of characters, efficient binarization and enhancement technique followed by a suitable connected component analysis procedure, text string detection from natural scenes by structure - based partition and grouping, and a robust algorithm for text detection in images. It is assumed that characters have closed contours, and a character string consists of characters which lie on a straight line in most cases. Therefore, by extracting closed contours and searching neighbors of them, character string regions can be extracted; Image binarization successfully processed natural scene images having shadows, non - uniform illumination, low contrast and large signal - dependent noise. Connected component analysis is used to define the final binary images that mainly consist of text regions. One technique chooses the candidate text characters from connected components by gradient feature and color feature. The text line grouping method performs Hough transform to fit text line among the centroids of text candidates. Each fitte d text line describes the orientation of a potential text string. The detected text string is presented by a rectangle region coveri ng all characters whose centroids are cascaded in its text line. To improve efficiency and accuracy, our algorithms are carried out in multi - scales. The proposed methods outperform the state - of - the - art results on the public Robust Reading Dataset, which contains text only in horizontal orientation. Furthermore, the effectiveness of our methods to detect text strings with arbitrary orientations is evaluated on the Oriented Scene Text Dataset collected by ourselves containing text strings in no horizontal orientations

Luminance cues constrain chromatic blur discrimination in natural scene stimuli

Introducing blur into the color components of a natural scene has very little effect on its percept, whereas blur introduced into the luminance component is very noticeable. Here we quantify the dominance of luminance information in blur detection and examine a number of potential causes. We show that the interaction between chromatic and luminance information is not explained by reduced acuity or spatial resolution limitations for chromatic cues, the effective contrast of the luminance cue, or chromatic and achromatic statistical regularities in the images. Regardless of the quality of chromatic information, the visual system gives primacy to luminance signals when determining edge location. In natural viewing, luminance information appears to be specialized for detecting object boundaries while chromatic information may be used to determine surface properties