Approche probabiliste hybride pour la recherche d'images par le contenu avec pondération des caractéristiques

Durant la dernière décennie, des quantités énormes de documents visuels (images et vidéos) sont produites chaque jour par les scientifiques, les journalistes, les amateurs, etc. Cette quantité a vite démontré la limite des systèmes de recherche d'images par mots clés, d'où la naissance du paradigme qu'on nomme Système de Recherche d'Images par le Contenu, en anglais Content-Based Image Retrieval (CBIR). Ces systèmes visent à localiser les images similaires à une requête constituée d'une ou plusieurs images, à l'aide des caractéristiques visuelles telles que la couleur, la forme et la texture. Ces caractéristiques sont dites de bas-niveau car elles ne reflètent pas la sémantique de l'image. En d'autres termes deux images sémantiquement différentes peuvent produire des caractéristiques bas-niveau similaires. Un des principaux défis de cette nouvelle vision des systèmes est l'organisation de la collection d'images pour avoir un temps de recherche acceptable. Pour faire face à ce défi, les techniques développées pour l'indexation des bases de données textuelles telles que les arbres sont massivement utilisées. Ces arbres ne sont pas adaptés aux données de grandes dimensions, comme c'est le cas des caractéristiques de bas-niveau des images. Dans ce mémoire, nous nous intéressons à ce défi. Nous introduisons une nouvelle approche probabiliste hybride pour l'organisation des collections d'images. Sur une collection d'images organisée hiérarchiquement en noeuds selon la sémantique des images, nous utilisons une approche générative pour l'estimation des mélanges de probabilités qui représentent l'apparence visuelle de chaque noeud dans la collection. Ensuite nous appliquons une approche discriminative pour l'estimation des poids des caractéristiques visuelles. L'idée dans notre travail, est de limiter la recherche seulement aux noeuds qui représentent mieux la sémantique de la requête, ce qui donne une propriété sémantique à la recherche et diminue le fossé sémantique causé par les caractéristiques de bas-niveau

Deep learning for content-based image retrieval: A comprehensive study

Learning effective feature representations and similarity measures are crucial to the retrieval performance of a content-based image retrieval (CBIR) system. Despite extensive research efforts for decades, it remains one of the most challenging open problems that considerably hinders the successes of real-world CBIR sys-tems. The key challenge has been attributed to the well-known “se-mantic gap ” issue that exists between low-level image pixels cap-tured by machines and high-level semantic concepts perceived by human. Among various techniques, machine learning has been ac-tively investigated as a possible direction to bridge the semantic gap in the long term. Inspired by recent successes of deep learning tech-niques for computer vision and other applications, in this paper, we attempt to address an open problem: if deep learning is a hope for bridging the semantic gap in CBIR and how much improvements i

Approximation and Relaxation Approaches for Parallel and Distributed Machine Learning

Large scale machine learning requires tradeoffs. Commonly this tradeoff has led practitioners to choose simpler, less powerful models, e.g. linear models, in order to process more training examples in a limited time. In this work, we introduce parallelism to the training of non-linear models by leveraging a different tradeoff--approximation. We demonstrate various techniques by which non-linear models can be made amenable to larger data sets and significantly more training parallelism by strategically introducing approximation in certain optimization steps. For gradient boosted regression tree ensembles, we replace precise selection of tree splits with a coarse-grained, approximate split selection, yielding both faster sequential training and a significant increase in parallelism, in the distributed setting in particular. For metric learning with nearest neighbor classification, rather than explicitly train a neighborhood structure we leverage the implicit neighborhood structure induced by task-specific random forest classifiers, yielding a highly parallel method for metric learning. For support vector machines, we follow existing work to learn a reduced basis set with extremely high parallelism, particularly on GPUs, via existing linear algebra libraries. We believe these optimization tradeoffs are widely applicable wherever machine learning is put in practice in large scale settings. By carefully introducing approximation, we also introduce significantly higher parallelism and consequently can process more training examples for more iterations than competing exact methods. While seemingly learning the model with less precision, this tradeoff often yields noticeably higher accuracy under a restricted training time budget

Face modeling for face recognition in the wild.

Face understanding is considered one of the most important topics in computer vision field since the face is a rich source of information in social interaction. Not only does the face provide information about the identity of people, but also of their membership in broad demographic categories (including sex, race, and age), and about their current emotional state. Facial landmarks extraction is the corner stone in the success of different facial analyses and understanding applications. In this dissertation, a novel facial modeling is designed for facial landmarks detection in unconstrained real life environment from different image modalities including infra-red and visible images. In the proposed facial landmarks detector, a part based model is incorporated with holistic face information. In the part based model, the face is modeled by the appearance of different face part(e.g., right eye, left eye, left eyebrow, nose, mouth) and their geometric relation. The appearance is described by a novel feature referred to as pixel difference feature. This representation is three times faster than the state-of-art in feature representation. On the other hand, to model the geometric relation between the face parts, the complex Bingham distribution is adapted from the statistical community into computer vision for modeling the geometric relationship between the facial elements. The global information is incorporated with the local part model using a regression model. The model results outperform the state-of-art in detecting facial landmarks. The proposed facial landmark detector is tested in two computer vision problems: boosting the performance of face detectors by rejecting pseudo faces and camera steering in multi-camera network. To highlight the applicability of the proposed model for different image modalities, it has been studied in two face understanding applications which are face recognition from visible images and physiological measurements for autistic individuals from thermal images. Recognizing identities from faces under different poses, expressions and lighting conditions from a complex background is an still unsolved problem even with accurate detection of landmark. Therefore, a learning similarity measure is proposed. The proposed measure responds only to the difference in identities and filter illuminations and pose variations. similarity measure makes use of statistical inference in the image plane. Additionally, the pose challenge is tackled by two new approaches: assigning different weights for different face part based on their visibility in image plane at different pose angles and synthesizing virtual facial images for each subject at different poses from single frontal image. The proposed framework is demonstrated to be competitive with top performing state-of-art methods which is evaluated on standard benchmarks in face recognition in the wild. The other framework for the face understanding application, which is a physiological measures for autistic individual from infra-red images. In this framework, accurate detecting and tracking Superficial Temporal Arteria (STA) while the subject is moving, playing, and interacting in social communication is a must. It is very challenging to track and detect STA since the appearance of the STA region changes over time and it is not discriminative enough from other areas in face region. A novel concept in detection, called supporter collaboration, is introduced. In support collaboration, the STA is detected and tracked with the help of face landmarks and geometric constraint. This research advanced the field of the emotion recognition

Kernel-based distance metric learning for content-based image retrieval

For a specific set of features chosen for representing images, the performance of a content-based image retrieval (CBIR) system depends critically on the similarity or dissimilarity measure used. Instead of manually choosing a distance function in advance, a more promising approach is to learn a good distance function from data automatically. In this paper, we propose a kernel approach to improve the retrieval performance of CBIR systems by learning a distance metric based on pairwise constraints between images as supervisory information. Unlike most existing metric learning methods which learn a Mahalanobis metric corresponding to performing linear transformation in the original image space, we define the transformation in the kernel-induced feature space which is nonlinearly related to the image space. Experiments performed on two real-world image databases show that our method not only improves the retrieval performance of Euclidean distance without distance learning, but it also outperforms other distance learning methods significantly due to its higher flexibility in metric learning. (c) 2006 Elsevier B.V. All rights reserved

Local selection of features and its applications to image search and annotation

In multimedia applications, direct representations of data objects typically involve hundreds or thousands of features. Given a query object, the similarity between the query object and a database object can be computed as the distance between their feature vectors. The neighborhood of the query object consists of those database objects that are close to the query object. The semantic quality of the neighborhood, which can be measured as the proportion of neighboring objects that share the same class label as the query object, is crucial for many applications, such as content-based image retrieval and automated image annotation. However, due to the existence of noisy or irrelevant features, errors introduced into similarity measurements are detrimental to the neighborhood quality of data objects. One way to alleviate the negative impact of noisy features is to use feature selection techniques in data preprocessing. From the original vector space, feature selection techniques select a subset of features, which can be used subsequently in supervised or unsupervised learning algorithms for better performance. However, their performance on improving the quality of data neighborhoods is rarely evaluated in the literature. In addition, most traditional feature selection techniques are global, in the sense that they compute a single set of features across the entire database. As a consequence, the possibility that the feature importance may vary across different data objects or classes of objects is neglected. To compute a better neighborhood structure for objects in high-dimensional feature spaces, this dissertation proposes several techniques for selecting features that are important to the local neighborhood of individual objects. These techniques are then applied to image applications such as content-based image retrieval and image label propagation. Firstly, an iterative K-NN graph construction method for image databases is proposed. A local variant of the Laplacian Score is designed for the selection of features for individual images. Noisy features are detected and sparsified iteratively from the original standardized feature vectors. This technique is incorporated into an approximate K-NN graph construction method so as to improve the semantic quality of the graph. Secondly, in a content-based image retrieval system, a generalized version of the Laplacian Score is used to compute different feature subspaces for images in the database. For online search, a query image is ranked in the feature spaces of database images. Those database images for which the query image is ranked highly are selected as the query results. Finally, a supervised method for the local selection of image features is proposed, for refining the similarity graph used in an image label propagation framework. By using only the selected features to compute the edges leading from labeled image nodes to unlabeled image nodes, better annotation accuracy can be achieved. Experimental results on several datasets are provided in this dissertation, to demonstrate the effectiveness of the proposed techniques for the local selection of features, and for the image applications under consideration