Re-ranking for Multimedia Indexing and Retrieval

Question Answering / NLPInternational audienceWe proposed a re-ranking method for improving the performance of semantic video indexing and retrieval. Experimental results show that the proposed re-ranking method is effective and it improves the system performance on average by about 16-22\% on TRECVID 2010 semantic indexing task

Towards training-free refinement for semantic indexing of visual media

Indexing of visual media based on content analysis has now moved beyond using individual concept detectors and there is now a fo- cus on combining concepts or post-processing the outputs of individual concept detection. Due to the limitations and availability of training cor- pora which are usually sparsely and imprecisely labeled, training-based refinement methods for semantic indexing of visual media suffer in cor- rectly capturing relationships between concepts, including co-occurrence and ontological relationships. In contrast to training-dependent methods which dominate this field, this paper presents a training-free refinement (TFR) algorithm for enhancing semantic indexing of visual media based purely on concept detection results, making the refinement of initial con- cept detections based on semantic enhancement, practical and flexible. This is achieved using global and temporal neighbourhood information inferred from the original concept detections in terms of weighted non- negative matrix factorization and neighbourhood-based graph propaga- tion, respectively. Any available ontological concept relationships can also be integrated into this model as an additional source of external a priori knowledge. Experiments on two datasets demonstrate the efficacy of the proposed TFR solution

Fusing semantics, observability, reliability and diversity of concept detectors for video search

ABSTRACT Effective utilization of semantic concept detectors for largescale video search has recently become a topic of intensive studies. One of main challenges is the selection and fusion of appropriate detectors, which considers not only semantics but also the reliability of detectors, observability and diversity of detectors in target video domains. In this paper, we present a novel fusion technique which considers different aspects of detectors for query answering. In addition to utilizing detectors for bridging the semantic gap of user queries and multimedia data, we also address the issue of "observability gap" among detectors which could not be directly inferred from semantic reasoning such as using ontology. To facilitate the selection of detectors, we propose the building of two vector spaces: semantic space (SS) and observability space (OS). We categorize the set of detectors selected separately from SS and OS into four types: anchor, bridge, positive and negative concepts. A multi-level fusion strategy is proposed to novelly combine detectors, allowing the enhancement of detector reliability while enabling the observability, semantics and diversity of concepts being utilized for query answering. By experimenting the proposed approach on TRECVID 2005-2007 datasets and queries, we demonstrate the significance of considering observability, reliability and diversity, in addition to the semantics of detectors to queries

情報検索における意味的ギャップの解消 : トピックモデルを用いた先進的画像探索

Tohoku University徳山豪課