Video browsing interfaces and applications: a review

We present a comprehensive review of the state of the art in video browsing and retrieval systems, with special emphasis on interfaces and applications. There has been a significant increase in activity (e.g., storage, retrieval, and sharing) employing video data in the past decade, both for personal and professional use. The ever-growing amount of video content available for human consumption and the inherent characteristics of video data—which, if presented in its raw format, is rather unwieldy and costly—have become driving forces for the development of more effective solutions to present video contents and allow rich user interaction. As a result, there are many contemporary research efforts toward developing better video browsing solutions, which we summarize. We review more than 40 different video browsing and retrieval interfaces and classify them into three groups: applications that use video-player-like interaction, video retrieval applications, and browsing solutions based on video surrogates. For each category, we present a summary of existing work, highlight the technical aspects of each solution, and compare them against each other

Temporal hybridity: Mixing live video footage with instant replay in real time

Copyright @ 2010 ACMIn this paper we explore the production of streaming media that involves live and recorded content. To examine this, we report on how the production practices and process are conducted through an empirical study of the production of live television, involving the use of live and non-live media under highly time critical conditions. In explaining how this process is managed both as an individual and collective activity, we develop the concept of temporal hybridity to explain the properties of these kinds of production system and show how temporally separated media are used, understood and coordinated. Our analysis is examined in the light of recent developments in computing technology and we present some design implications to support amateur video production.The research was partly made possible by a grant from the Swedish Governmental Agency for Innovation Systems to the Mobile Life VinnExcellence Center, in partnership with SonyEricsson, Ericsson, Microsoft Research, Nokia Research, TeliaSonera and the City of Stockholm

Multi-modal surrogates for retrieving and making sense of videos: is synchronization between the multiple modalities optimal?

Video surrogates can help people quickly make sense of the content of a video before downloading or seeking more detailed information. Visual and audio features of a video are primary information carriers and might become important components of video retrieval and video sense-making. In the past decades, most research and development efforts on video surrogates have focused on visual features of the video, and comparatively little work has been done on audio surrogates and examining their pros and cons in aiding users' retrieval and sense-making of digital videos. Even less work has been done on multi-modal surrogates, where more than one modality are employed for consuming the surrogates, for example, the audio and visual modalities. This research examined the effectiveness of a number of multi-modal surrogates, and investigated whether synchronization between the audio and visual channels is optimal. A user study was conducted to evaluate six different surrogates on a set of six recognition and inference tasks to answer two main research questions: (1) How do automatically-generated multi-modal surrogates compare to manually-generated ones in video retrieval and video sense-making? and (2) Does synchronization between multiple surrogate channels enhance or inhibit video retrieval and video sense-making? Forty-eight participants participated in the study, in which the surrogates were measured on the the time participants spent on experiencing the surrogates, the time participants spent on doing the tasks, participants' performance accuracy on the tasks, participants' confidence in their task responses, and participants' subjective ratings on the surrogates. On average, the uncoordinated surrogates were more helpful than the coordinated ones, but the manually-generated surrogates were only more helpful than the automatically-generated ones in terms of task completion time. Participants' subjective ratings were more favorable for the coordinated surrogate C2 (Magic A + V) and the uncoordinated surrogate U1 (Magic A + Storyboard V) with respect to usefulness, usability, enjoyment, and engagement. The post-session questionnaire comments demonstrated participants' preference for the coordinated surrogates, but the comments also revealed the value of having uncoordinated sensory channels

Indexing, browsing and searching of digital video

Video is a communications medium that normally brings together moving pictures with a synchronised audio track into a discrete piece or pieces of information. The size of a “piece ” of video can variously be referred to as a frame, a shot, a scene, a clip, a programme or an episode, and these are distinguished by their lengths and by their composition. We shall return to the definition of each of these in section 4 this chapter. In modern society, video is ver

The TRECVID 2007 BBC rushes summarization evaluation pilot

This paper provides an overview of a pilot evaluation of video summaries using rushes from several BBC dramatic series. It was carried out under the auspices of TRECVID. Twenty-two research teams submitted video summaries of up to 4% duration, of 42 individual rushes video files aimed at compressing out redundant and insignificant material. The output of two baseline systems built on straightforward content reduction techniques was contributed by Carnegie Mellon University as a control. Procedures for developing ground truth lists of important segments from each video were developed at Dublin City University and applied to the BBC video. At NIST each summary was judged by three humans with respect to how much of the ground truth was included, how easy the summary was to understand, and how much repeated material the summary contained. Additional objective measures included: how long it took the system to create the summary, how long it took the assessor to judge it against the ground truth, and what the summary's duration was. Assessor agreement on finding desired segments averaged 78% and results indicate that while it is difficult to exceed the performance of baselines, a few systems did

To Archive or Not to Archive: The Resistant Potential of Digital Poetry

This essay addresses the much discussed problem of archiving digital poetry. Digital media are labile, and several writers of digital poetry are incorporating the media’s ephemerality into their poetics. Rather than rehash arguments that have been taking place within the field of digital media and digital poetics for years, I turn to the field of contemporary art curation and preservation, a field in which curators and archivists are struggling with the very immediate concerns, ethical and otherwise, related to archiving works that are made from ephemeral media. One particular digital poem that has recently broken, has recently become unreadable, is Talan Memmott’s Lexia to Perplexia. Memmott composed the poem in 2000, and he incorporated the poem’s inevitable obsolescence into the text of the poem itself. He has since refused to “fix” or “update” the poem, because he contends that that would make it something other than what it was intended to be. Rather, he is choosing to let the poem die because that is what the poem is supposed to do. This essay concludes with a discussion of the political implications of acknowledging the ephemerality of digital media, the resistant potential of the poem when its ephemerality is embraced, and some ways in which archivists can preserve the memory of the poem without necessarily preserving the poem itself

Digital libraries: What do users want?

This is the post-print version of the Article of the Article. The official published version can be accessed from the link below - Copyright @ 2006 EmeraldPurpose – The purpose of this study is to determine user suggestions for digital libraries' functionality and features. Design/methodology/approach – A survey was conducted as part of this study, in which users' suggestions for digital libraries were solicited, as well as their ranking opinions on a range of suggested digital library features. Findings – The study revealed that, regardless of users' information technology (IT) backgrounds, their expectations of digital libraries' functionality are the same. However, based on users' previous experiences with digital libraries, their requirements with respect to specific features may change. Practical implications – Involving users in digital library design should be an integral step in the process of building a digital library – in addition to the classic roles of evaluation and testing. Originality/value – In previous digital library user studies, users were involved implicitly (e.g. observed) or explicitly (e.g. diary notes). However, they were never asked to suggest digital library features or functionalities, as this was left to usability and domain experts. This study approached digital library design from a new perspective, giving users an opportunity to express their suggestions on future functionality and features of digital libraries. Moreover, in contrast to previous work, this study has explicitly taken into account the IT abilities of those interacting with a digital library

Digital libraries for creative communities

Digital library technologies have a great deal to offer to creative, design communities. They can enable large collections of text, images, music, video and other information objects to be organised and accessed in interesting and diverse ways. Ordinary people—people not traditionally viewed as 'creators' or 'designers'—can now conceive, assemble, build, and disseminate new information collections. This paper explores the development rationale behind the Greenstone digital library technology. We also examine three examples of creative new techniques for accessing and presenting information in digital libraries and stress the importance of tailoring information access to support the requirements of the users and application area

Planar Refrains

My practice explores phenomenal poetic truths that exist in fissures between the sensual and physical qualities of material constructs. Magnifying this confounding interspace, my work activates specific instruments within mutable, relational systems of installation, movement, and documentation. The tools I fabricate function within variable orientations and are implemented as both physical barriers and thresholds into alternate, virtual domains. Intersecting fragments of sound and moving image build a nexus of superimposed spatialities, while material constructions are enveloped in ephemeral intensities. Within this compounded environment, both mind and body are charged as active sites through which durational, contemplative experiences can pass. Reverberation, the ghostly refrain of a sound calling back to our ears from a distant plane, can intensify our emotional experience of place. My project Planar Refrains utilizes four electro-mechanical reverb plates, analog audio filters designed to simulate expansive acoustic arenas. Historically these devices have provided emotive voicings to popular studio recordings, dislocating the performer from the commercial studio and into a simulated reverberant territory of mythic proportions. The material resonance of steel is used to filter a recorded signal, shaping the sound of a human performance into something more transformative, a sound embodying otherworldly dynamics. In subverting the designed utility of reverb plates, I am exploring their value as active surfaces extending across different spatial realities. The background of ephemeral sonic residue is collapsed into the foreground, a filter becomes sculpture, and this sculpture becomes an instrument in an evolving soundscape