About optimal use of color points of interest for content-based image retrieval

In content-based image retrieval systems, the main approaches based on the query-by-example paradigm involve an approximate search on the whole image, which requires a global description of it. When considering tasks like object recognition or partial queries on particular area, these methods become inadequate and more local characterizations must be employed. In this context, image description based on points of interest appear best adapted. The point characterization which proved reliable is based on invariants to rotation and in particular on combinations of the Hilbert's differential invariants. For gray value images, such a description used to be considered up to third order at least. More recently, generalizations to color images were proposed for stereovision and image retrieval. Some of them propose to consider the invariants only at first order and to enrich the characterization with geometrical constraints for describing spatial relations between points, while others consider higher order invariants and compute some combinations of them to achieve illumination changes invariance. In this report, we discuss the advantages and drawbacks of these different choices, with the aim of proposing an optimal use of color points of interest for content-based image indexing and retrieval

What's beyond query by example?

Over the last ten years, the crucial problem of information retrieval in multimedia documents has boosted research activities in the field of visual appearance indexing and retrieval by content. In the early research years, the concept of the «query by visual example» (QBVE) has been proposed and shown to be relevant for visual information retrieval. It is obvious that QBVE is not able to satisfy the multiple visual search usage requirements. In this paper, we focus on two major approaches that correspond to two different retrieval paradigms. First, we present the partial visual query that ignores the background of the images and allows a straight user expression on its visual interest without relevance feedback mechanism. The second retrieval paradigm consists in searching for the user mental image when no starting visual example is available. query by logical composition of region categories when a visual thesaurus is generated. This new approach relies on the unpervised generation of a visual thesaurus from which query by logical composition of region categories can be performed. This query paradigm is closely related to that of text retrieval. Mental image search is a challenging and promising issue for retrieval by visual content in the forthcoming years since it allows different rich user expression and interaction modes with the search engine

Labelling the Behaviour of Local Descriptors for Selective Video Content Retrieval

This paper presents an approach for indexing a large set of videos by considering the cinematic behaviour of local visual features along the sequences. The proposed concept is based on the extraction and the local description of interest points and further on the estimation of their trajectories along the video sequence. Analysing the low-level description obtained allows to highlight semantic trends of behaviours and then to assign labels. Such an indexing approach of the video content has several interesting properties: the low-level description provides a rich and compact description, while labels of behaviour provide a generic and semantic description, relevant for selective video content retrieval depending on the application. The approach is firstly evaluated for Content-Based Copy Detection. We show that taking these labels into account allows to significantly reduce false alarms. Secondly, the approach is experimented on particular applications of video monitoring, where selective labels of behaviour show their capability to improve the analysis and the retrieval of spatio-temporal video content

Mise en correspondance d'images en couleur : Application à la synthèse de vues intermédiaires

Image matching is a key step for many computer vision applications. Among all the approaches existing for grayscale images, the iconic methods provide the most convincing results. However, none of them take advantage of the richness of the color, while their very essence is to exploit the most of the information contained in the image signal. The main contribution of the work developed in this manuscript is then to propose a more robust matching method than those encountered to date, using the additional information contained in the color content. This approach is innovative at several levels of the matching process. Thus, as a first contribution, we present a class of point of interest detectors specific to color. Two new operators are proposed, evaluated and proved to be more stable than the previously known grey level detectors. Color also allows to set up a local and robust method for describing these points locally. This second contribution is based on the Hilbert differential invariants calculated only at order one, thanks to the contribution of color information. It is invariant to Euclidean transformations of the image, and we also propose an original method that makes it invariant to changes in illumination. The strong complexity of classical matching methods make them unusable with large sets of points. Our last contribution consists in integrating the color primitives thus characterized into a new matching process, made effective against large sets of points by the use of robust geometric constraints regardless of image transformations.La mise en correspondance d'images est une étape clé pour un grand nombre d'applications de vision par ordinateur. Parmi toutes les approches existantes en niveau de gris, les méthodes iconiques fournissent les résultats les plus probants. Cependant, aucune d'entre elles ne tire profit de la richesse de la couleur, alors que leur essence même est d'exploiter au maximum l'information contenue dans le signal de l'image. La principale contribution des travaux développés dans ce mémoire consiste alors à mettre en place une méthode de mise en correspondance plus robuste que celles rencontrées jusqu'à ce jour, en mettant en jeu l'information supplémentaire que contiennent les images en couleur. Cette approche est innovante à plusieurs niveaux du processus d'appariement. Ainsi, nous présentons dans un premier traitement une classe de détecteurs de points d'intérêt spécifique à la couleur. Deux nouveaux opérateurs sont proposés, évalués et se révèlent plus stables que les détecteurs jusqu'alors connus en niveau de gris. La couleur nous permet également de mettre en place une méthode de caractérisation de ces points, locale et robuste. Celle-ci est basée sur les invariants différentiels de Hilbert calculés seulement à l'ordre un, grâce à l'apport de l'information couleur. Elle est donc invariante aux transformations euclidiennes de l'image ; nous proposons également une méthode originale qui la rend invariante aux changements d'illumination. La forte combinatoire des méthodes d'appariement classiques rendent ces dernières inexploitables avec beaucoup de points. Nos travaux consistent en outre à intégrer les primitives couleur ainsi caractérisées dans un nouveau processus de mise en correspondance, rendu efficace face aux grands ensembles de points par l'utilisation de contraintes géométriques robustes quelles que soient les transformations de l'image

Recherche par contenu visuel dans les grandes collections d’images

International audienc

Index structures for image search by content: A survey. In "A state of the art on Computation Intensive Methods" report, MUSCLE European Network of Excellence (FP 6), WP 7, July

National audienc

Image representation, book chapter, Encyclopedia of Database Systems:Multimedia Databases

resum

Object-based queries using color points of interest

International audienc