Similarity search and data mining techniques for advanced database systems.

Modern automated methods for measurement, collection, and analysis of data in industry and science are providing more and more data with drastically increasing structure complexity. On the one hand, this growing complexity is justified by the need for a richer and more precise description of real-world objects, on the other hand it is justified by the rapid progress in measurement and analysis techniques that allow the user a versatile exploration of objects. In order to manage the huge volume of such complex data, advanced database systems are employed. In contrast to conventional database systems that support exact match queries, the user of these advanced database systems focuses on applying similarity search and data mining techniques. Based on an analysis of typical advanced database systems — such as biometrical, biological, multimedia, moving, and CAD-object database systems — the following three challenging characteristics of complexity are detected: uncertainty (probabilistic feature vectors), multiple instances (a set of homogeneous feature vectors), and multiple representations (a set of heterogeneous feature vectors). Therefore, the goal of this thesis is to develop similarity search and data mining techniques that are capable of handling uncertain, multi-instance, and multi-represented objects. The first part of this thesis deals with similarity search techniques. Object identification is a similarity search technique that is typically used for the recognition of objects from image, video, or audio data. Thus, we develop a novel probabilistic model for object identification. Based on it, two novel types of identification queries are defined. In order to process the novel query types efficiently, we introduce an index structure called Gauss-tree. In addition, we specify further probabilistic models and query types for uncertain multi-instance objects and uncertain spatial objects. Based on the index structure, we develop algorithms for an efficient processing of these query types. Practical benefits of using probabilistic feature vectors are demonstrated on a real-world application for video similarity search. Furthermore, a similarity search technique is presented that is based on aggregated multi-instance objects, and that is suitable for video similarity search. This technique takes multiple representations into account in order to achieve better effectiveness. The second part of this thesis deals with two major data mining techniques: clustering and classification. Since privacy preservation is a very important demand of distributed advanced applications, we propose using uncertainty for data obfuscation in order to provide privacy preservation during clustering. Furthermore, a model-based and a density-based clustering method for multi-instance objects are developed. Afterwards, original extensions and enhancements of the density-based clustering algorithms DBSCAN and OPTICS for handling multi-represented objects are introduced. Since several advanced database systems like biological or multimedia database systems handle predefined, very large class systems, two novel classification techniques for large class sets that benefit from using multiple representations are defined. The first classification method is based on the idea of a k-nearest-neighbor classifier. It employs a novel density-based technique to reduce training instances and exploits the entropy impurity of the local neighborhood in order to weight a given representation. The second technique addresses hierarchically-organized class systems. It uses a novel hierarchical, supervised method for the reduction of large multi-instance objects, e.g. audio or video, and applies support vector machines for efficient hierarchical classification of multi-represented objects. User benefits of this technique are demonstrated by a prototype that performs a classification of large music collections. The effectiveness and efficiency of all proposed techniques are discussed and verified by comparison with conventional approaches in versatile experimental evaluations on real-world datasets

Efficient Analysis in Multimedia Databases

The rapid progress of digital technology has led to a situation where computers have become ubiquitous tools. Now we can find them in almost every environment, be it industrial or even private. With ever increasing performance computers assumed more and more vital tasks in engineering, climate and environmental research, medicine and the content industry. Previously, these tasks could only be accomplished by spending enormous amounts of time and money. By using digital sensor devices, like earth observation satellites, genome sequencers or video cameras, the amount and complexity of data with a spatial or temporal relation has gown enormously. This has led to new challenges for the data analysis and requires the use of modern multimedia databases. This thesis aims at developing efficient techniques for the analysis of complex multimedia objects such as CAD data, time series and videos. It is assumed that the data is modeled by commonly used representations. For example CAD data is represented as a set of voxels, audio and video data is represented as multi-represented, multi-dimensional time series. The main part of this thesis focuses on finding efficient methods for collision queries of complex spatial objects. One way to speed up those queries is to employ a cost-based decompositioning, which uses interval groups to approximate a spatial object. For example, this technique can be used for the Digital Mock-Up (DMU) process, which helps engineers to ensure short product cycles. This thesis defines and discusses a new similarity measure for time series called threshold-similarity. Two time series are considered similar if they expose a similar behavior regarding the transgression of a given threshold value. Another part of the thesis is concerned with the efficient calculation of reverse k-nearest neighbor (RkNN) queries in general metric spaces using conservative and progressive approximations. The aim of such RkNN queries is to determine the impact of single objects on the whole database. At the end, the thesis deals with video retrieval and hierarchical genre classification of music using multiple representations. The practical relevance of the discussed genre classification approach is highlighted with a prototype tool that helps the user to organize large music collections. Both the efficiency and the effectiveness of the presented techniques are thoroughly analyzed. The benefits over traditional approaches are shown by evaluating the new methods on real-world test datasets

Multimodal Pattern Matching for Audio-Visual Query and Retrieval

A necessary capability for content-based retrieval is to support the paradigm of query by example. In the past, there have been several attempts to use low-level features for video retrieval. None of the approaches however uses the multimedia information content of the video. We present an algorithm for matching multimodal (audio-visual) patterns for the purpose of content-based video retrieval. The novel ability of our approach to use the information content in multiple media coupled with a strong emphasis on temporal similarity differentiates it from the state-of-the -art in content-based retrieval. At the core of the pattern matching scheme is a dynamic programming algorithm, which leads to a significant improvement in performance. Coupling the use of audio with video this algorithm can be applied to grouping of shots based on audio-visual similarity. This is much more effective in constructing scenes from shots than using only visual content to do the same

Comparaison des documents audiovisuels<br />par Matrice de Similarité

The work of this thesis relates to the comparison of video documents. The field of digital video is in full expansion. Videos are now present in large quantity even for personal use. The video comparison is a basic analysis operation in complement of classification, extraction and structuring of videos.Traditional approaches of comparison are primarily based on the low-level features of the videos to be compared, considered as multidimensional vectors. Other approaches are based on the similarity of frames without taking into account neither the temporal composition of the video nor the audiolayer. The main disadvantage of these methods is that they reduce the comparison role to a simple operator robust to noise effects. Such operators are generally used in order to identify the various specimens of a same document.The originality of our approach lies in the introduction of the of style similarity notion, taking as a starting point the human criteria into the comparison. These criteria are more flexible, and do not impose a strict similarity of all the studied features at the same time.We define an algorithm of extraction of the similarities between the series of values produced bythe analysis of the audiovisual low-level features. The algorithm is inspired by the dynamic programmingand the time series comparison methods.We propose a representation of the data resulting from these processings in the form of a matrixpattern suitable for the visual and immediate comparison of two videos. This matrix is then used topropose a generic similarity measure. The measure is applicable independently to videos of comparableor heterogeneous contents.We developed several applications to demonstrate the behavior of the comparison method and thesimilarity measure. The experiments concern primarily: - the identification of the structure in acollection/sub-collection of documents, - the description of stylistics elements in a movie, and - theanalysis of the grid of programs from a TV stream.Les travaux de cette thèse concernent la comparaison des documents vidéo. Dans le domaine en pleine expansion de la vidéo numérique, les documents disponibles sont maintenant présents en quantité importante même dans les foyers. Opération de base de tout type d'analyse de contenus, en complément de la classification, de l'extraction et de la structuration, la comparaison dans le domaine de l'audiovisuel est d'une utilité qui n'est pas à démontrer.Des approches classiques de comparaison se basent essentiellement sur l'ensemble des caractéristiquesbas niveaux des documents à comparer, en les considérant comme des vecteurs multidimensionnels. D'autres approches se basent sur la similarité des images composant la vidéo sans tenir compte de la composition temporelle du document ni de la bandeson. Le défaut que l'on peut reprocher à ces méthodes est qu'elles restreignent la comparaison à un simple opérateur binaire robuste au bruit. De tels opérateurs sont généralement utilisés afin d'identifier les différents exemplaires d'un même document. L'originalité de notre démarche réside dans le fait que nous introduisons la notion de la similarité de styleen s'inspirant des critères humains dans la comparaison des documents vidéo. Ces critèressont plus souples, et n'imposent pas une similarité stricte de toutes les caractéristiques étudiéesà la fois.En nous inspirant de la programmation dynamique et de la comparaison des séries chronologiques, nous définissons un algorithme d'extraction des similarités entre les séries de valeurs produites par l'analyse de caractéristiques audiovisuelles de bas-niveau. Ensuite, un second traitement générique approxime le résultat de l'algorithme de la longueur de la PlusLongue Sous-Séquence Commune (PLSC) plus rapidement que ce dernier. Nous proposons une représentation des données issues de ces traitements sous la forme d'un schéma matriciel propre à la comparaison visuelle et immédiate de deux contenus. Cette matrice peut être également utilisée pour définir une mesure de similarité générique, applicable à des documents de même genre ou de genres hétérogènes.Plusieurs applications ont été mises en place pour démontrer le comportement de la méthode de comparaison et de la mesure de similarité, ainsi que leur pertinence. Les expérimentations concernent essentiellement : - l'identification d'une structure organisationnelle en collection / sous-collection d'une base de documents, - la mise en évidence d'élémentsstylistiques dans un film de cinéma, - la mise en évidence de la grille de programmes d'unflux de télévision