Exploiting multimedia in creating and analysing multimedia Web archives

The data contained on the web and the social web are inherently multimedia and consist of a mixture of textual, visual and audio modalities. Community memories embodied on the web and social web contain a rich mixture of data from these modalities. In many ways, the web is the greatest resource ever created by human-kind. However, due to the dynamic and distributed nature of the web, its content changes, appears and disappears on a daily basis. Web archiving provides a way of capturing snapshots of (parts of) the web for preservation and future analysis. This paper provides an overview of techniques we have developed within the context of the EU funded ARCOMEM (ARchiving COmmunity MEMories) project to allow multimedia web content to be leveraged during the archival process and for post-archival analysis. Through a set of use cases, we explore several practical applications of multimedia analytics within the realm of web archiving, web archive analysis and multimedia data on the web in general

Mind the Gap: Another look at the problem of the semantic gap in image retrieval

This paper attempts to review and characterise the problem of the semantic gap in image retrieval and the attempts being made to bridge it. In particular, we draw from our own experience in user queries, automatic annotation and ontological techniques. The first section of the paper describes a characterisation of the semantic gap as a hierarchy between the raw media and full semantic understanding of the media's content. The second section discusses real users' queries with respect to the semantic gap. The final sections of the paper describe our own experience in attempting to bridge the semantic gap. In particular we discuss our work on auto-annotation and semantic-space models of image retrieval in order to bridge the gap from the bottom up, and the use of ontologies, which capture more semantics than keyword object labels alone, as a technique for bridging the gap from the top down

Terrestrial applications: An intelligent Earth-sensing information system

For Abstract see A82-2214

'Girlfriends and Strawberry Jam’: Tagging Memories, Experiences, and Events for Future Retrieval

In this short paper we have some preliminary thoughts about tagging everyday life events in order to allow future retrieval of events or experiences related to events. Elaboration of these thoughts will be done in the context of the recently started Network of Excellence PetaMedia (Peer-to-Peer Tagged Media) and the Network of Excellence SSPNet (Social Signal Processing), to start in 2009, both funded by the European Commission's Seventh Framework Programme. Descriptions of these networks will be given later in this paper

Video information retrieval using objects and ostensive relevance feedback

In this paper, we present a brief overview of current approaches to video information retrieval (IR) and we highlight its limitations and drawbacks in terms of satisfying user needs. We then describe a method of incorporating object-based relevance feedback into video IR which we believe opens up new possibilities for helping users find information in video archives. Following this we describe our own work on shot retrieval from video archives which uses object detection, object-based relevance feedback and a variation of relevance feedback called ostensive RF which is particularly appropriate for this type of retrieval

Detection of setting and subject information in documentary video

Interpretation of video information is a difficult task for computer vision and machine intelligence. In this paper we examine the utility of a non-image based source of information about video contents, namely the shot list, and study its use in aiding image interpretation. We show how the shot list may be analysed to produce a simple summary of the \u27who and where\u27 of a documentary or interview video. In order to detect the subject of a video we use the notion of a \u27shot syntax\u27 of a particular genre to isolate actual interview sections

Searching Data: A Review of Observational Data Retrieval Practices in Selected Disciplines

A cross-disciplinary examination of the user behaviours involved in seeking and evaluating data is surprisingly absent from the research data discussion. This review explores the data retrieval literature to identify commonalities in how users search for and evaluate observational research data. Two analytical frameworks rooted in information retrieval and science technology studies are used to identify key similarities in practices as a first step toward developing a model describing data retrieval

Bridging the Semantic Gap in Multimedia Information Retrieval: Top-down and Bottom-up approaches

Semantic representation of multimedia information is vital for enabling the kind of multimedia search capabilities that professional searchers require. Manual annotation is often not possible because of the shear scale of the multimedia information that needs indexing. This paper explores the ways in which we are using both top-down, ontologically driven approaches and bottom-up, automatic-annotation approaches to provide retrieval facilities to users. We also discuss many of the current techniques that we are investigating to combine these top-down and bottom-up approaches

Symbiosis between the TRECVid benchmark and video libraries at the Netherlands Institute for Sound and Vision

Audiovisual archives are investing in large-scale digitisation efforts of their analogue holdings and, in parallel, ingesting an ever-increasing amount of born- digital files in their digital storage facilities. Digitisation opens up new access paradigms and boosted re-use of audiovisual content. Query-log analyses show the shortcomings of manual annotation, therefore archives are complementing these annotations by developing novel search engines that automatically extract information from both audio and the visual tracks. Over the past few years, the TRECVid benchmark has developed a novel relationship with the Netherlands Institute of Sound and Vision (NISV) which goes beyond the NISV just providing data and use cases to TRECVid. Prototype and demonstrator systems developed as part of TRECVid are set to become a key driver in improving the quality of search engines at the NISV and will ultimately help other audiovisual archives to offer more efficient and more fine-grained access to their collections. This paper reports the experiences of NISV in leveraging the activities of the TRECVid benchmark

Off-line processing of ERS-1 synthetic aperture radar data with high precision and high throughput

The first European remote sensing satellite ERS-1 will be launched by the European Space Agency (ESA) in 1989. The expected lifetime is two to three years. The spacecraft sensors will primarily support ocean investigations and to a limited extent also land applications. Prime sensor is the Active Microwave Instrumentation (AMI) operating in C-Band either as Synthetic Aperture Radar (SAR) or as Wave-Scatterometer and simultaneously as Wind-Scatterometer. In Europe there will be two distinct types of processing for ERS-1 SAR data, Fast Delivery Processing and Precision Processing. Fast Delivery Proceessing will be carried out at the ground stations and up to three Fast Delivery products per pass will be delivered to end users via satellite within three hours after data acquisition. Precision Processing will be carried out in delayed time and products will not be generated until several days or weeks after data acquisition. However, a wide range of products will be generated by several Processing and Archiving Facilities (PAF) in a joint effort coordinated by ESA. The German Remote Sensing Data Center (Deutsches Fernerkundungsdatenzentrum DFD) will develop and operate one of these facilities. The related activities include the acquisition, processing and evaluation of such data for scientific, public and commercial users. Based on this experience the German Remote Sensing Data Center is presently performing a Phase-B study regarding the development of a SAR processor for ERS-1. The conceptual design of this processing facility is briefly outlined