Deformable Prototypes for Encoding Shape Categories in Image Databases

We describe a method for shape-based image database search that uses deformable prototypes to represent categories. Rather than directly comparing a candidate shape with all shape entries in the database, shapes are compared in terms of the types of nonrigid deformations (differences) that relate them to a small subset of representative prototypes. To solve the shape correspondence and alignment problem, we employ the technique of modal matching, an information-preserving shape decomposition for matching, describing, and comparing shapes despite sensor variations and nonrigid deformations. In modal matching, shape is decomposed into an ordered basis of orthogonal principal components. We demonstrate the utility of this approach for shape comparison in 2-D image databases.Office of Naval Research (Young Investigator Award N00014-06-1-0661

EsPRESSo: Efficient Privacy-Preserving Evaluation of Sample Set Similarity

Electronic information is increasingly often shared among entities without complete mutual trust. To address related security and privacy issues, a few cryptographic techniques have emerged that support privacy-preserving information sharing and retrieval. One interesting open problem in this context involves two parties that need to assess the similarity of their datasets, but are reluctant to disclose their actual content. This paper presents an efficient and provably-secure construction supporting the privacy-preserving evaluation of sample set similarity, where similarity is measured as the Jaccard index. We present two protocols: the first securely computes the (Jaccard) similarity of two sets, and the second approximates it, using MinHash techniques, with lower complexities. We show that our novel protocols are attractive in many compelling applications, including document/multimedia similarity, biometric authentication, and genetic tests. In the process, we demonstrate that our constructions are appreciably more efficient than prior work.Comment: A preliminary version of this paper was published in the Proceedings of the 7th ESORICS International Workshop on Digital Privacy Management (DPM 2012). This is the full version, appearing in the Journal of Computer Securit

Indexation et recherche d'images

Cet article présente les problèmes et les amorces de solutions posés par la recherche d'images dans des bases d'images dès lors que l'on souhaite une indexation automatique, comme c'est le cas pour des documents écrits. Les outils utilisés pour caractériser des images et permettre une recherche de ce type sont actuellement frustes, et souvent reposent sur des caractéristiques globales utilisant largement l'information de luminance. Nous plaidons pour l'usage de caractéristiques locales bien qu'ils aient l'inconvénient de se heurter à des problèmes de segmentation. Nous montrons, par un exemple détaillé, que des éléments de solution existent et qu'ils peuvent indiquer la voie pour des recherches futures. Cet article illustre aussi l'intérêt que présentent les déjà anciennes recherches en reconnaissance des formes pour ce genre de problèmes

Classification of Marine Vessels in a Littoral Environment Using a Novel Training Database

Research into object classification has led to the creation of hundreds of databases for use as training sets in object classification algorithms. Datasets made up of thousands of cars, people, boats, faces and everyday objects exist for general classification techniques. However, no commercially available database exists for use with detailed classification and categorization of marine vessels commonly found in littoral environments. This research seeks to fill this void and is the combination of a multi-stage research endeavor designed to provide the missing marine vessel ontology. The first of the two stages performed to date introduces a novel training database called the Lister Littoral Database 900 (LLD-900) made up of over 900 high-quality images. These images consist of high-resolution color photos of marine vessels in working, active conditions taken directly from the field and edited for best possible use. Segmentation masks of each boat have been developed to separate the image into foreground and background sections. Segmentation masks that include boat wakes as part of the foreground section are the final image type included. These are included to allow for wake affordance detection algorithms rely on the small changes found in wakes made by different moving vessels. Each of these three types of images are split into their respective general classification folders, which consist of a differing number of boat categories dependent on the research stage. In the first stage of research, the initial database is tested using a simple, readily available classification algorithm known as the Nearest Neighbor Classifier. The accuracy of the database as a training set is tested and recorded and potential improvements are documented. The second stage incorporates these identified improvements and reconfigures the database before retesting the modifications using the same Nearest Neighbor Classifier along with two new methods known as the K-Nearest Neighbor Classifier and the Min-Mean Distance Classifier. These additional algorithms are also readily available and offer basic classification testing using different classification techniques. Improvements in accuracy are calculated and recorded. Finally, further improvements for a possible third iteration are discussed. The goal of this research is to establish the basis for a training database to be used with classification algorithms to increase the security of ports, harbors, shipping channels and bays. The purpose of the database is to train existing and newly created algorithms to properly identify and classify all boats found in littoral areas so that anomalous behavior detection techniques can be applied to determine when a threat is present. This research represents the completion of the initial steps in accomplishing this goal delivering a novel framework for use with littoral area marine vessel classification. The completed work is divided and presented in two separate papers written specifically for submission to and publication at appropriate conferences. When fully integrated with computer vision techniques, the database methodology and ideas presented in this thesis research will help to provide a vital new level of security in the littoral areas around the world

Foveated object recognition by corner search

textHere we describe a gray scale object recognition system based on foveated corner finding, the computation of sequential fixation points, and elements of Lowe’s SIFT transform. The system achieves rotational, transformational, and limited scale invariant object recognition that produces recognition decisions using data extracted from sequential fixation points. It is broken into two logical steps. The first is to develop principles of foveated visual search and automated fixation selection to accomplish corner search. The result is a new algorithm for finding corners which is also a corner-based algorithm for aiming computed foveated visual fixations. In the algorithm, long saccades move the fovea to previously unexplored areas of the image, while short saccades improve the accuracy of putative corner locations. The system is tested on two natural scenes. As an interesting comparison study we compare fixations generated by the algorithm with those of subjects viewing the same images, whose eye movements are being recorded by an eyetracker. The comparison of fixation patterns is made using an information-theoretic measure. Results show that the algorithm is a good locator of corners, but does not correlate particularly well with human visual fixations. The second step is to use the corners located, which meet certain goodness criteria, as keypoints in a modified version of the SIFT algorithm. Two scales are implemented. This implementation creates a database of SIFT features of known objects. To recognize an unknown object, a corner is located and a feature vector created. The feature vector is compared with those in the database of known objects. The process is continued for each corner in the unknown object until enough information has been accumulated to reach a decision. The system was tested on 78 gray scale objects, hand tools and airplanes, and shown to perform well.Electrical and Computer Engineerin

Combining crowd worker, algorithm, and expert efforts to find boundaries of objects in images

While traditional approaches to image analysis have typically relied upon either manual annotation by experts or purely-algorithmic approaches, the rise of crowdsourcing now provides a new source of human labor to create training data or perform computations at run-time. Given this richer design space, how should we utilize algorithms, crowds, and experts to better annotate images? To answer this question for the important task of finding the boundaries of objects or regions in images, I focus on image segmentation, an important precursor to solving a variety of fundamental image analysis problems, including recognition, classification, tracking, registration, retrieval, and 3D visualization. The first part of the work includes a detailed analysis of the relative strengths and weaknesses of three different approaches to demarcate object boundaries in images: by experts, by crowdsourced laymen, and by automated computer vision algorithms. The second part of the work describes three hybrid system designs that integrate computer vision algorithms and crowdsourced laymen to demarcate boundaries in images. Experiments revealed that hybrid system designs yielded more accurate results than relying on algorithms or crowd workers alone and could yield segmentations that are indistinguishable from those created by biomedical experts. To encourage community-wide effort to continue working on developing methods and systems for image-based studies which can have real and measurable impact that benefit society at large, datasets and code are publicly-shared (http://www.cs.bu.edu/~betke/BiomedicalImageSegmentation/)