UTwente does Brave New Tasks for MediaEval 2012: Searching and Hyperlinking

In this paper we report our experiments and results for the brave new searching and hyperlinking tasks for the MediaEval Benchmark Initiative 2012. The searching task involves nding target video segments based on a short natural language sentence query and the hyperlinking task involves nding links from the target video segments to other related video segments in the collection using a set of anchor segments in the videos that correspond to the textual search queries. To nd the starting points in the video, we only used speech transcripts and metadata as evidence source, however, other visual features (for e.g., faces, shots and keyframes) might also aect results for a query. We indexed speech transcripts and metadata, furthermore, the speech transcripts were indexed at speech segment level and at sentence level to improve the likelihood of nding jump-in-points. For linking video segments, we computed k-nearest neighbours of video segments using euclidean distance

Access to recorded interviews: A research agenda

Recorded interviews form a rich basis for scholarly inquiry. Examples include oral histories, community memory projects, and interviews conducted for broadcast media. Emerging technologies offer the potential to radically transform the way in which recorded interviews are made accessible, but this vision will demand substantial investments from a broad range of research communities. This article reviews the present state of practice for making recorded interviews available and the state-of-the-art for key component technologies. A large number of important research issues are identified, and from that set of issues, a coherent research agenda is proposed

Towards Affordable Disclosure of Spoken Word Archives

This paper presents and discusses ongoing work aiming at affordable disclosure of real-world spoken word archives in general, and in particular of a collection of recorded interviews with Dutch survivors of World War II concentration camp Buchenwald. Given such collections, the least we want to be able to provide is search at different levels and a flexible way of presenting results. Strategies for automatic annotation based on speech recognition – supporting e.g., within-document search– are outlined and discussed with respect to the Buchenwald interview collection. In addition, usability aspects of the spoken word search are discussed on the basis of our experiences with the online Buchenwald web portal. It is concluded that, although user feedback is generally fairly positive, automatic annotation performance is still far from satisfactory, and requires additional research

Linking inside a video collection - what and how to measure?

Although linking video to additional information sources seems to be a sensible approach to satisfy information needs of user, the perspective of users is not yet analyzed on a fundamental level in real-life scenarios. However, a better understanding of the motivation of users to follow links in video, which anchors users prefer to link from within a video, and what type of link targets users are typically interested in, is important to be able to model automatic linking of audiovisual content appropriately. In this paper we report on our methodology towards eliciting user requirements with respect to video linking in the course of a broader study on user requirements in searching and a series of benchmark evaluations on searching and linking

AXES at TRECVid 2011

Abstract The AXES project participated in the interactive known-item search task (KIS) and the interactive instance search task (INS) for TRECVid 2011. We used the same system architecture and a nearly identical user interface for both the KIS and INS tasks. Both systems made use of text search on ASR, visual concept detectors, and visual similarity search. The user experiments were carried out with media professionals and media students at the Netherlands Institute for Sound and Vision, with media professionals performing the KIS task and media students participating in the INS task. This paper describes the results and findings of our experiments

Affordable access to multimedia by exploiting collateral data

In addition to multimedia collections and their metadata,\ud there often is a variety of collateral data sources available on\ud (parts of) a collection. Collateral data – secondary information\ud objects that relate to the primary multimedia documents\ud – can be very useful in the process of automated generation of\ud annotations for multimedia archives in that they reduce both\ud costs and effort in annotation and access. Furthermore, they\ud can be used to enhance result presentation in retrieval engines.\ud To optimally exploit collateral data, methods for automatic indexing\ud as well as changes in the current archiving workflow\ud are proposed

Speech Indexing

This chapter will focus on the automatic extraction of information from the speech in multimedia documents. This approach is often referred to as speech indexing and it can be regarded as a subfield of audio indexing that also incorporates for example the analysis of music and sounds. If the objective of the recognition of the words spoken is to support retrieval, one commonly speaks of spoken document retrieval (SDR). If the objective is on the coupling of various media types the term media mining or even cross-media mining is used. Most attention in this chapter will go to SDR. The focus is less on searching (an index of ) a multimedia database, but on enabling multiple views on the data by cross-linking all the available multifaceted information sources in a multimedia database. In section 1.6 cross-media mining will be discussed in more detail

Exploiting Program Guides for Contextualisation

Archives of cultural heritage organisations typically consist of collections in various formats (e.g. photos, video, texts) that are inherently related. Often, such disconnected collections represent value in itself but effectuating links between 'core' and 'context' collection items in various levels of granularity could result in a 'one-plus-one-makes-three' scenario both from a contextualisation perspective (public presentations, research) and access perspective. A key issue is the identification of contextual objects that can be associated with objects in the core collections, or the other way around. Traditionally, such associations have been created manually. For most organizations however, this approach does not scale. In this paper, we describe a case in which a semi-automatic approach was employed to create contextual links between television broadcast schedules in program guides (context collection) and the programs in the archive (core collection) of a large audiovisual heritage organisation