Multimedia search without visual analysis: the value of linguistic and contextual information

This paper addresses the focus of this special issue by analyzing the potential contribution of linguistic content and other non-image aspects to the processing of audiovisual data. It summarizes the various ways in which linguistic content analysis contributes to enhancing the semantic annotation of multimedia content, and, as a consequence, to improving the effectiveness of conceptual media access tools. A number of techniques are presented, including the time-alignment of textual resources, audio and speech processing, content reduction and reasoning tools, and the exploitation of surface features

combining multimodal external resources for event-based news video retrieval and question answering

Ph.DDOCTOR OF PHILOSOPH

Surface features in video retrieval

This paper assesses the usefulness of surface features in a multimedia retrieval setting. Surface features describe the metadata or structure of a document rather than the content. We note that the distribution of these features varies across topics. The paper shows how these distributions can be obtained through relevance feedback and how this allows for adaptation of (content-based) search results for topic or user preference. An analysis of the distribution of surface features in the TRECVID collection indicates that they are potentially useful, and a preliminary feedback experiment confirms that exploiting surface features can improve retrieval effectiveness

A multi-resolution multi-source and multi-modal (M3) transductive framework for concept detection in news video

Ph.DDOCTOR OF PHILOSOPH

A DOMAIN-CENTRIC APPROACH TO DESIGNING USER INTERFACES OF VIDEO RETRIEVAL SYSTEMS

Thesis (PhD) - Indiana University, Information Science, 2007User- and task-centric efforts in video information retrieval (IR) research are needed because current experiments are showing few significant results. It is our belief that unsatisfactory results in video IR can be partially attributed to the overemphasis on technologically-driven approaches to interface development and system evaluation. This study explored variables that have been consistently overlooked in video retrieval efforts, including those related to domain and search tasks. The underlying goal of this study is to promote alternative means for evaluating video retrieval systems, and to make progress toward developing new design principles and a video seeking model. A series of interactive search runs were conducted using a video retrieval system called ViewFinder. ViewFinder was implemented to search and browse the NASA K - 16 Science Education Programs. The system includes new design features that take into account the unique characteristics of the domain and associated tasks. Users with a background in Science Education, including teachers and academic majors, were recruited to perform a number of search tasks. Results from the search experiments were collected and analyzed using both objective and subjective measures. From these results, researchers gained further knowledge about domain-centric video search tasks, including how textual, visual, and hybrid tasks were all deemed important by science educators. Further analysis of experimental results also revealed associations between search tasks, user interaction, interface features and functions, and system effectiveness. The evaluation of individual interface features and functions exhibited that keyword searching was significant for retrieving Science Education video. However, these experiments also produced positive results for various visual search features. Unlike keyword searching, which was consistent and effective across many task types, the use and effectiveness of visual search and browse features were shown to be task dependent. Overall, the results from this study highlight the importance of user- and task-centric methods in video retrieval, as they provided researchers with additional understanding of the influences of domain-specific search tasks on user interaction with video systems. In addition, the experimental methodology employed for this study encourages future foundations for developing and evaluating video search interfaces designed for specific domains and search tasks

Multi-graph based active learning for interactive video retrieval

Master'sMASTER OF SCIENC