310 research outputs found
Video Indexing and Retrieval Techniques Using Novel Approaches to Video Segmentation, Characterization, and Similarity Matching
Multimedia applications are rapidly spread at an ever-increasing rate introducing a number of challenging problems at the hands of the research community, The most significant and influential problem, among them, is the effective access to stored data. In spite of the popularity of keyword-based search technique in alphanumeric databases, it is inadequate for use with multimedia data due to their unstructured nature. On the other hand, a number of content-based access techniques have been developed in the context of image indexing and retrieval; meanwhile video retrieval systems start to gain wide attention, This work proposes a number of techniques constituting a fully content-based system for retrieving video data. These techniques are primarily targeting the efficiency, reliability, scalability, extensibility, and effectiveness requirements of such applications. First, an abstract representation of the video stream, known as the DC sequence, is extracted. Second, to deal with the problem of video segmentation, an efficient neural network model is introduced. The novel use of the neural network improves the reliability while the efficiency is achieved through the instantaneous use of the recall phase to identify shot boundaries. Third, the problem of key frames extraction is addressed using two efficient algorithms that adapt their selection decisions based on the amount of activity found in each video shot enabling the selection of a near optimal expressive set of key frames. Fourth, the developed system employs an indexing scheme that supports two low-level features, color and texture, to represent video data, Finally, we propose, in the retrieval stage, a novel model for performing video data matching task that integrates a number of human-based similarity factors. All our software implementations are in Java, which enables it to be used across heterogeneous platforms. The retrieval system performance has been evaluated yielding a very good retrieval rate and accuracy, which demonstrate the effectiveness of the developed system
Audiovisual processing for sports-video summarisation technology
In this thesis a novel audiovisual feature-based scheme is proposed for the automatic summarization of sports-video content The scope of operability of the scheme is designed to encompass the wide variety o f sports genres that come under the description âfield-sportsâ. Given the assumption that, in terms of conveying the narrative of a field-sports-video, score-update events constitute the most significant moments, it is proposed that their detection should thus yield a favourable summarisation solution. To this end, a generic methodology is proposed for the automatic identification of score-update events in field-sports-video content. The scheme is based on the development of robust extractors for a set of critical features, which are shown to reliably indicate their locations. The evidence gathered by the feature extractors is combined and analysed using a Support Vector Machine (SVM), which performs the event detection process. An SVM is chosen on the basis that its underlying technology represents an implementation of the latest generation of machine learning algorithms, based on the recent advances in statistical learning. Effectively, an SVM offers a solution to optimising the classification performance of a decision hypothesis, inferred from a given set of training data. Via a learning phase that utilizes a 90-hour field-sports-video trainmg-corpus, the SVM infers a score-update event model by observing patterns in the extracted feature evidence. Using a similar but distinct 90-hour evaluation corpus, the effectiveness of this model is then tested genencally across multiple genres of fieldsports- video including soccer, rugby, field hockey, hurling, and Gaelic football. The results suggest that in terms o f the summarization task, both high event retrieval and content rejection statistics are achievable
Recommended from our members
From content-based to semantic image retrieval. Low level feature extraction, classification using image processing and neural networks, content based image retrieval, hybrid low level and high level based image retrieval in the compressed DCT domain.
Digital image archiving urgently requires advanced techniques for more efficient storage and retrieval methods because of the increasing amount of digital. Although JPEG supply systems to compress image data efficiently, the problems of how to organize the image database structure for efficient indexing and retrieval, how to index and retrieve image data from DCT compressed domain and how to interpret image data semantically are major obstacles for further development of digital image database system. In content-based image, image analysis is the primary step to extract useful information from image databases. The difficulty in content-based image retrieval is how to summarize the low-level features into high-level or semantic descriptors to facilitate the retrieval procedure. Such a shift toward a semantic visual data learning or detection of semantic objects generates an urgent need to link the low level features with semantic understanding of the observed visual information. To solve such a -semantic gapÂż problem, an efficient way is to develop a number of classifiers to identify the presence of semantic image components that can be connected to semantic descriptors. Among various semantic objects, the human face is a very important example, which is usually also the most significant element in many images and photos. The presence of faces can usually be correlated to specific scenes with semantic inference according to a given ontology. Therefore, face detection can be an efficient tool to annotate images for semantic descriptors. In this thesis, a paradigm to process, analyze and interpret digital images is proposed. In order to speed up access to desired images, after accessing image data, image features are presented for analysis. This analysis gives not only a structure for content-based image retrieval but also the basic units
ii
for high-level semantic image interpretation. Finally, images are interpreted and classified into some semantic categories by semantic object detection categorization algorithm
An object-based approach to retrieval of image and video content
Promising new directions have been opened up for content-based visual retrieval in recent years. Object-based retrieval which allows users to manipulate video objects as part of their searching and browsing interaction, is one of these. It is the purpose of this thesis to constitute itself as a part of a larger stream of research that investigates visual objects as a possible approach to advancing the use of semantics in content-based visual retrieval.
The notion of using objects in video retrieval has been seen as desirable for some years, but only very recently has technology started to allow even very basic object-location functions on video. The main hurdles to greater use of objects in video retrieval are the overhead of
object segmentation on large amounts of video and the issue of whether objects can actually be used efficiently for multimedia retrieval. Despite this, there are already some examples of work which supports retrieval based on video objects.
This thesis investigates an object-based approach to content-based visual retrieval. The main research contributions of this work are a study of shot boundary detection on compressed domain video where a fast detection approach is proposed and evaluated, and a study on the use of objects in interactive image retrieval. An object-based retrieval framework is developed in order to investigate object-based retrieval on a corpus of natural image
and video. This framework contains the entire processing chain required to analyse, index and interactively retrieve images and video via object-to-object matching. The experimental results indicate that object-based searching consistently outperforms image-based search using low-level features. This result goes some way towards validating the approach of allowing users to select objects as a basis for searching video archives when the information need dictates it as appropriate
Construction de mosaïques de super-résolution à partir de la vidéo de basse résolution. Application au résumé vidéo et la dissimulation d'erreurs de transmission.
La numĂ©risation des vidĂ©os existantes ainsi que le dĂ©veloppement explosif des services multimĂ©dia par des rĂ©seaux comme la diffusion de la tĂ©lĂ©vision numĂ©rique ou les communications mobiles ont produit une Ă©norme quantitĂ© de vidĂ©os compressĂ©es. Ceci nĂ©cessite des outils dâindexation et de navigation efficaces, mais une indexation avant lâencodage nâest pas habituelle. Lâapproche courante est le dĂ©codage complet des ces vidĂ©os pour ensuite crĂ©er des indexes. Ceci est trĂšs coĂ»teux et par consĂ©quent non rĂ©alisable en temps rĂ©el. De plus, des informations importantes comme le mouvement, perdus lors du dĂ©codage, sont reestimĂ©es bien que dĂ©jĂ prĂ©sentes dans le flux comprimĂ©. Notre but dans cette thĂšse est donc la rĂ©utilisation des donnĂ©es dĂ©jĂ prĂ©sents dans le flux comprimĂ© MPEG pour lâindexation et la navigation rapide. Plus prĂ©cisĂ©ment, nous extrayons des coefficients DC et des vecteurs de mouvement. Dans le cadre de cette thĂšse, nous nous sommes en particulier intĂ©ressĂ©s Ă la construction de mosaĂŻques Ă partir des images DC extraites des images I. Une mosaĂŻque est construite par recalage et fusion de toutes les images dâune sĂ©quence vidĂ©o dans un seul systĂšme de coordonnĂ©es. Ce dernier est en gĂ©nĂ©ral alignĂ© avec une des images de la sĂ©quence : lâimage de rĂ©fĂ©rence. Il en rĂ©sulte une seule image qui donne une vue globale de la sĂ©quence. Ainsi, nous proposons dans cette thĂšse un systĂšme complet pour la construction des mosaĂŻques Ă partir du flux MPEG-1/2 qui tient compte de diffĂ©rentes problĂšmes apparaissant dans des sĂ©quences vidĂ©o rĂ©eles, comme par exemple des objets en mouvment ou des changements dâĂ©clairage. Une tĂąche essentielle pour la construction dâune mosaĂŻque est lâestimation de mouvement entre chaque image de la sĂ©quence et lâimage de rĂ©fĂ©rence. Notre mĂ©thode se base sur une estimation robuste du mouvement global de la camĂ©ra Ă partir des vecteurs de mouvement des images P. Cependant, le mouvement global de la camĂ©ra estimĂ© pour une image P peut ĂȘtre incorrect car il dĂ©pend fortement de la prĂ©cision des vecteurs encodĂ©s. Nous dĂ©tectons les images P concernĂ©es en tenant compte des coefficients DC de lâerreur encodĂ©e associĂ©e et proposons deux mĂ©thodes pour corriger ces mouvements. UnemosaĂŻque construite Ă partir des images DC a une rĂ©solution trĂšs faible et souffre des effets dâaliasing dus Ă la nature des images DC. Afin dâaugmenter sa rĂ©solution et dâamĂ©liorer sa qualitĂ© visuelle, nous appliquons une mĂ©thode de super-rĂ©solution basĂ©e sur des rĂ©tro-projections itĂ©ratives. Les mĂ©thodes de super-rĂ©solution sont Ă©galement basĂ©es sur le recalage et la fusion des images dâune sĂ©quence vidĂ©o, mais sont accompagnĂ©es dâune restauration dâimage. Dans ce cadre, nous avons dĂ©veloppĂ© une nouvellemĂ©thode dâestimation de flou dĂ» au mouvement de la camĂ©ra ainsi quâune mĂ©thode correspondante de restauration spectrale. La restauration spectrale permet de traiter le flou globalement, mais, dans le cas des obvi jets ayant un mouvement indĂ©pendant du mouvement de la camĂ©ra, des flous locaux apparaissent. Câest pourquoi, nous proposons un nouvel algorithme de super-rĂ©solution dĂ©rivĂ© de la restauration spatiale itĂ©rative de Van Cittert et Jansson permettant de restaurer des flous locaux. En nous basant sur une segmentation dâobjets en mouvement, nous restaurons sĂ©parĂ©ment lamosaĂŻque dâarriĂšre-plan et les objets de lâavant-plan. Nous avons adaptĂ© notre mĂ©thode dâestimation de flou en consĂ©quence. Dans une premier temps, nous avons appliquĂ© notre mĂ©thode Ă la construction de rĂ©sumĂ© vidĂ©o avec pour lâobjectif la navigation rapide par mosaĂŻques dans la vidĂ©o compressĂ©e. Puis, nous Ă©tablissions comment la rĂ©utilisation des rĂ©sultats intermĂ©diaires sert Ă dâautres tĂąches dâindexation, notamment Ă la dĂ©tection de changement de plan pour les images I et Ă la caractĂ©risation dumouvement de la camĂ©ra. Enfin, nous avons explorĂ© le domaine de la rĂ©cupĂ©ration des erreurs de transmission. Notre approche consiste en construire une mosaĂŻque lors du dĂ©codage dâun plan ; en cas de perte de donnĂ©es, lâinformation manquante peut ĂȘtre dissimulĂ©e grace Ă cette mosaĂŻque
Object Tracking
Object tracking consists in estimation of trajectory of moving objects in the sequence of images. Automation of the computer object tracking is a difficult task. Dynamics of multiple parameters changes representing features and motion of the objects, and temporary partial or full occlusion of the tracked objects have to be considered. This monograph presents the development of object tracking algorithms, methods and systems. Both, state of the art of object tracking methods and also the new trends in research are described in this book. Fourteen chapters are split into two sections. Section 1 presents new theoretical ideas whereas Section 2 presents real-life applications. Despite the variety of topics contained in this monograph it constitutes a consisted knowledge in the field of computer object tracking. The intention of editor was to follow up the very quick progress in the developing of methods as well as extension of the application
Detection of near-duplicates in large image collections
The vast numbers of images on the Web include many duplicates, and an even larger number of near-duplicate variants derived from the same original. These include thumbnails stored by search engines, copies shared by various news portals, and images that appear on multiple web sites, legitimately or otherwise. Such near-duplicates appear in the results of many web image searches, and constitute redundancy, and may also represent infringements of copyright. Digital images can be easily altered through simple digital manipulation such as conversion to grey-scale, colour balance change, rescaling, rotation, and cropping. Any of these operations defeat simple duplicate detection methods such as bit-level hashing. The ability to detect such variants with a reasonable degree of reliability and accuracy would support reduction of redundancy in collections and in presentation of search results, and also allow detection of possible copyright violations. Some existing methods for identifying near-duplicates are derived from computer vision techniques; these have shown high effectiveness for this domain, but are computationally expensive, and therefore impractical for large image collections. Other methods address the problem using conventional CBIR approaches that are more efficient but are typically not as robust. None of the previous methods have addressed the problem in its entirety, and none have addressed the large scale near-duplicate problem on the Web; there has been no analysis of the kinds of alterations that are common on the Web, nor any or evaluation of whether real cases of near-duplication can in fact be identified. In this thesis, we analyse the different types of alterations and near-duplicates existent in a range of popular web image searches, and establish a collection and evaluation ground truth using real-world near-duplicate examples. We present a simple ranking approach to reduce the number of local-descriptors, and therefore improve the efficiency of the descriptor-based retrieval method for near-duplicate detection. The descriptor-based method has been shown to produce near-perfect detection of near-duplicates, but was previously computationally very expensive. We show that while maintaining comparable effectiveness, our method scales well for large collections of hundreds of thousands of images. We also explore a more compact indexing structure to support near duplicate image detection. We develop a method to automatically detect the pair-wise near-duplicate relationship of images without the use of a query. We adapt the hash-based probabilistic counting method --- originally used for near-duplicate text document detection --- with the local descriptors; our adaptation offers the first effective and efficient non-query-based approach to this domain. We further incorporate our pair-wise detection approach for clustering of near-duplicates. We present a clustering method specifically for near-duplicate images, where our method is arguably the first clustering method to achieve a high level of effectiveness in this domain. We also show that near-duplicates within a large collection of a million images can be effectively clustered using our approach in less than an hour using relatively modest computational resources. Overall, our proposed methods provide practical approaches to the detection and management of near-duplicate images in large collection
Scene Segmentation and Object Classification for Place Recognition
This dissertation tries to solve the place recognition and loop closing problem in a way similar to human visual system. First, a novel image segmentation algorithm is developed. The image segmentation algorithm is based on a Perceptual Organization model, which allows the image segmentation algorithm to âperceiveâ the special structural relations among the constituent parts of an unknown object and hence to group them together without object-specific knowledge.
Then a new object recognition method is developed. Based on the fairly accurate segmentations generated by the image segmentation algorithm, an informative object description that includes not only the appearance (colors and textures), but also the parts layout and shape information is built. Then a novel feature selection algorithm is developed. The feature selection method can select a subset of features that best describes the characteristics of an object class. Classifiers trained with the selected features can classify objects with high accuracy.
In next step, a subset of the salient objects in a scene is selected as landmark objects to label the place. The landmark objects are highly distinctive and widely visible. Each landmark object is represented by a list of SIFT descriptors extracted from the object surface. This object representation allows us to reliably recognize an object under certain viewpoint changes. To achieve efficient scene-matching, an indexing structure is developed. Both texture feature and color feature of objects are used as indexing features. The texture feature and the color feature are viewpoint-invariant and hence can be used to effectively find the candidate objects with similar surface characteristics to a query object.
Experimental results show that the object-based place recognition and loop detection method can efficiently recognize a place in a large complex outdoor environment
- âŠ