Local image representations using pruned salient points with applications to CBIR

Salient points are locations in an image where there is a significant variation with respect to a chosen image feature. Since the set of salient points in an image capture important local characteristics of that image, they can form the basis of a good image representation for content-based image retrieval (CBIR). The features for a salient point should represent the local characteristic of that point so that the similarity between features indicates the similarity between the salient points. Traditional uses of salient points for CBIR assign features to a salient point based on the image features of all pixels in a window around that point. However, since salient points are often on the boundary of objects, the features assigned to a salient point often involve pixels from different objects. In this paper, we propose a CBIR system that uses a novel salient point method that both reduces the number of salient points using a segmentation as a filter, and also improves the representation so that it is a more faithful representation of a single object (or portion of an object) that includes information about its surroundings. We also introduce an improved Expectation Maximization-Diverse Density (EM-DD) based multiple-instance learning algorithm. Experimental results show that our CBIR techniques improve retrieval performance by ∼5%-11 % as compared with current methods

Organising and structuring a visual diary using visual interest point detectors

As wearable cameras become more popular, researchers are increasingly focusing on novel applications to manage the large volume of data these devices produce. One such application is the construction of a Visual Diary from an individual’s photographs. Microsoft’s SenseCam, a device designed to passively record a Visual Diary and cover a typical day of the user wearing the camera, is an example of one such device. The vast quantity of images generated by these devices means that the management and organisation of these collections is not a trivial matter. We believe wearable cameras, such as SenseCam, will become more popular in the future and the management of the volume of data generated by these devices is a key issue. Although there is a significant volume of work in the literature in the object detection and recognition and scene classification fields, there is little work in the area of setting detection. Furthermore, few authors have examined the issues involved in analysing extremely large image collections (like a Visual Diary) gathered over a long period of time. An algorithm developed for setting detection should be capable of clustering images captured at the same real world locations (e.g. in the dining room at home, in front of the computer in the office, in the park, etc.). This requires the selection and implementation of suitable methods to identify visually similar backgrounds in images using their visual features. We present a number of approaches to setting detection based on the extraction of visual interest point detectors from the images. We also analyse the performance of two of the most popular descriptors - Scale Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF).We present an implementation of a Visual Diary application and evaluate its performance via a series of user experiments. Finally, we also outline some techniques to allow the Visual Diary to automatically detect new settings, to scale as the image collection continues to grow substantially over time, and to allow the user to generate a personalised summary of their data