SVM Based Indoor/Mixed/Outdoor Classification for Digital Photo Annotation in a Ubiquitous Computing Environment

This paper extends our previous framework for digital photo annotation by adding noble approach of indoor/mixed/outdoor image classification. We propose the best feature vectors for a support vector machine based indoor/mixed/ outdoor image classification. While previous research classifies photographs into indoor and outdoor, this study extends into three types, including indoor, mixed, and outdoor classes. This three-class method improves the performance of outdoor classification. This classification scheme showed 5--10% higher performance than previous research. This method is one of the components for digital image annotation. A digital camera or an annotation server connected to a ubiquitous computing network can automatically annotate captured photos using the proposed method

Automatic indoor/outdoor scene classification

The advent and wide acceptance of digital imaging technology has motivated an upsurge in research focused on managing the ever-growing number of digital images. Current research in image manipulation represents a general shift in the field of computer vision from traditional image analysis based on low-level features (e.g. color and texture) to semantic scene understanding based on high-level features (e.g. grass and sky). One particular area of investigation is scene categorization, where the organization of a large number of images is treated as a classification problem. Generally, the classification involves mapping a set of traditional low-level features to semantically meaningful categories, such as indoor and outdoor scenes, using a classifier engine. Successful indoor/outdoor scene categorization is beneficial to a number of image manipulation applications, as indoor and outdoor scenes represent among the most general scene types. In content-based image retrieval, for example, a query for a scene containing a sunset can be restricted to images in the database pre-categorized as outdoor scenes. Also, in image enhancement, categorization of a scene as indoor vs. outdoor can lead to improved color balancing and tone reproduction. Prior research in scene classification has shown that high-level information can, in fact, be inferred from low-level image features. Classification rates of roughly 90% have been reported using low-level features to predict indoor scenes vs. outdoor scenes. However, the high classification rates are often achieved by using computationally expensive, high-dimensional feature sets, thus limiting the practical implementation of such systems. To address this problem, a low complexity, low-dimensional feature set was extracted in a variety of configurations in the work presented here. Due to their excellent generalization performance, Support Vector Machines (SVMs) were used to manage the tradeoff between reduced dimensionality and increased classification accuracy. It was determined that features extracted from image subblocks, as opposed to the full image, can yield better classification rates when combined in a second stage. In particular, applying SVMs in two stages led to an indoor/outdoor classification accuracy of 90.2% on a large database of consumer photographs provided by Kodak. Finally, it was also shown that low-level and semantic features can be integrated efficiently using Bayesian networks for increased accuracy. Specifically, the integration of grass and sky semantic features with color and texture low-level features increased the indoor/outdoor classification rate to 92.8% on the same database of images

Detecting the presence of large buildings in natural images

This paper addresses the issue of classification of lowlevel features into high-level semantic concepts for the purpose of semantic annotation of consumer photographs. We adopt a multi-scale approach that relies on edge detection to extract an edge orientation-based feature description of the image, and apply an SVM learning technique to infer the presence of a dominant building object in a general purpose collection of digital photographs. The approach exploits prior knowledge on the image context through an assumption that all input images are �outdoor�, i.e. indoor/outdoor classification (the context determination stage) has been performed. The proposed approach is validated on a diverse dataset of 1720 images and its performance compared with that of the MPEG-7 edge histogram descriptor

INDOOR-OUTDOOR IMAGE CLASSIFICATION USING DICHROMATIC REFLECTION MODEL AND HARALICK FEATURES

The problem of indoor-outdoor image classification using supervised learning is addressed in this paper. Conventional indoor-outdoor image classification methods, partition an image into predefined sub-blocks for feature extraction. However in this paper, we use a simple color segmentation stage to acquire meaningful regions from the image for feature extraction. The features that are used to describe an image are color correlated temperature, Haralick features, segment area and segment position. For the classification phase, an MLP was trained and tested using a dataset of 800 images. A classification accuracy of 94% compared with the result of other state of the art indoor-outdoor image classification methods showed the efficiency of the proposed method

EFFECTIVE COMBINING OF COLOR AND TEXTURE DESCRIPTORS FOR INDOOR-OUTDOOR IMAGE CLASSIFICATION

Although many indoor-outdoor image classification methods have been proposed in the literature, most of them have omitted comparison with basic methods to justify the need for complex feature extraction and classification procedures. In this paper we propose a relatively simple but highly accurate method for indoor-outdoor image classification, based on combination of carefully engineered MPEG-7 color and texture descriptors. In order to determine the optimal combination of descriptors which is characterized by efficient extraction, compact representation and high accuracy, we conducted comprehensive empirical tests over several color and texture descriptors. The optimal descriptors combination was used for training and testing of a binary SVM classifier. We have shown that the proper descriptors preprocessing before SVM classification has significant impact on the final result. Comprehensive experimental evaluation shows that the proposed method outperforms several more complex indoor-outdoor image classification techniques on a couple of public datasets

A hybrid technique for face detection in color images

In this paper, a hybrid technique for face detection in color images is presented. The proposed technique combines three analysis models, namely skin detection, automatic eye localization, and appearance-based face/nonface classification. Using a robust histogram-based skin detection model, skin-like pixels are first identified in the RGB color space. Based on this, face bounding-boxes are extracted from the image. On detecting a face bounding-box, approximate positions of the candidate mouth feature points are identified using the redness property of image pixels. A region-based eye localization step, based on the detected mouth feature points, is then applied to face bounding-boxes to locate possible eye feature points in the image. Based on the distance between the detected eye feature points, face/non-face classification is performed over a normalized search area using the Bayesian discriminating feature (BDF) analysis method. Some subjective evaluation results are presented on images taken using digital cameras and a Webcam, representing both indoor and outdoor scenes

Text-based approaches for non-topical image categorization

The rapid expansion of multimedia digital collections brings to the fore the need for classifying not only text documents but their embedded non-textual parts as well. We propose a model for basing classification of multimedia on broad, non-topical features, and show how information on targeted nearby pieces of text can be used to effectively classify photographs on a first such feature, distinguishing between indoor and outdoor images. We examine several variations to a TF*IDF-based approach for this task, empirically analyze their effects, and evaluate our system on a large collection of images from current news newsgroups. In addition, we investigate alternative classification and evaluation methods, and the effects that secondary features have on indoor/outdoor classification. Using density estimation over the raw TF*IDF values, we obtain a classification accuracy of 82%, a number that outperforms baseline estimates and earlier, image-based approaches, at least in the domain of news articles, and that nears the accuracy of humans who perform the same task with access to comparable information

Relating visual and semantic image descriptors

This paper addresses the automatic analysis of visual content and extraction of metadata beyond pure visual descriptors. Two approaches are described: Automatic Image Annotation (AIA) and Confidence Clustering (CC). AIA attempts to automatically classify images based on two binary classifiers and is designed for the consumer electronics domain. Contrastingly, the CC approach does not attempt to assign a unique label to images but rather to organise the database based on concepts

What do we perceive in a glance of a real-world scene?

What do we see when we glance at a natural scene and how does it change as the glance becomes longer? We asked naive subjects to report in a free-form format what they saw when looking at briefly presented real-life photographs. Our subjects received no specific information as to the content of each stimulus. Thus, our paradigm differs from previous studies where subjects were cued before a picture was presented and/or were probed with multiple-choice questions. In the first stage, 90 novel grayscale photographs were foveally shown to a group of 22 native-English-speaking subjects. The presentation time was chosen at random from a set of seven possible times (from 27 to 500 ms). A perceptual mask followed each photograph immediately. After each presentation, subjects reported what they had just seen as completely and truthfully as possible. In the second stage, another group of naive individuals was instructed to score each of the descriptions produced by the subjects in the first stage. Individual scores were assigned to more than a hundred different attributes. We show that within a single glance, much object- and scene-level information is perceived by human subjects. The richness of our perception, though, seems asymmetrical. Subjects tend to have a propensity toward perceiving natural scenes as being outdoor rather than indoor. The reporting of sensory- or feature-level information of a scene (such as shading and shape) consistently precedes the reporting of the semantic-level information. But once subjects recognize more semantic-level components of a scene, there is little evidence suggesting any bias toward either scene-level or object-level recognition