Video browsing interfaces and applications: a review

We present a comprehensive review of the state of the art in video browsing and retrieval systems, with special emphasis on interfaces and applications. There has been a significant increase in activity (e.g., storage, retrieval, and sharing) employing video data in the past decade, both for personal and professional use. The ever-growing amount of video content available for human consumption and the inherent characteristics of video data—which, if presented in its raw format, is rather unwieldy and costly—have become driving forces for the development of more effective solutions to present video contents and allow rich user interaction. As a result, there are many contemporary research efforts toward developing better video browsing solutions, which we summarize. We review more than 40 different video browsing and retrieval interfaces and classify them into three groups: applications that use video-player-like interaction, video retrieval applications, and browsing solutions based on video surrogates. For each category, we present a summary of existing work, highlight the technical aspects of each solution, and compare them against each other

Beyond 2D-grids: a dependence maximization view on image browsing

Ideally, one would like to perform image search using an intuitive and friendly approach. Many existing image search engines, however, present users with sets of images arranged in some default order on the screen, typically the relevance to a query, only. While this certainly has its advantages, arguably, a more flexible and intuitive way would be to sort images into arbitrary structures such as grids, hierarchies, or spheres so that images that are visually or semantically alike are placed together. This paper focuses on designing such a navigation system for image browsers. This is a challenging task because arbitrary layout structure makes it difficult -- if not impossible -- to compute cross-similarities between images and structure coordinates, the main ingredient of traditional layouting approaches. For this reason, we resort to a recently developed machine learning technique: kernelized sorting. It is a general technique for matching pairs of objects from different domains without requiring cross-domain similarity measures and hence elegantly allows sorting images into arbitrary structures. Moreover, we extend it so that some images can be preselected for instance forming the tip of the hierarchy allowing to subsequently navigate through the search results in the lower levels in an intuitive way

A content-based retrieval system for UAV-like video and associated metadata

In this paper we provide an overview of a content-based retrieval (CBR) system that has been specifically designed for handling UAV video and associated meta-data. Our emphasis in designing this system is on managing large quantities of such information and providing intuitive and efficient access mechanisms to this content, rather than on analysis of the video content. The retrieval unit in our system is termed a "trip". At capture time, each trip consists of an MPEG-1 video stream and a set of time stamped GPS locations. An analysis process automatically selects and associates GPS locations with the video timeline. The indexed trip is then stored in a shared trip repository. The repository forms the backend of a MPEG-211 compliant Web 2.0 application for subsequent querying, browsing, annotation and video playback. The system interface allows users to search/browse across the entire archive of trips and, depending on their access rights, to annotate other users' trips with additional information. Interaction with the CBR system is via a novel interactive map-based interface. This interface supports content access by time, date, region of interest on the map, previously annotated specific locations of interest and combinations of these. To develop such a system and investigate its practical usefulness in real world scenarios, clearly a significant amount of appropriate data is required. In the absence of a large volume of UAV data with which to work, we have simulated UAV-like data using GPS tagged video content captured from moving vehicles

Revisiting revisitation in computer interaction: organic bookmark management.

According to Milic-Frayling et al. (2004), there are two general ways of user browsing i.e. search (finding a website where the user has never visited before) and revisitation (returning to a website where the user has visited in the past). The issue of search is relevant to search engine technology, whilst revisitation concerns web usage and browser history mechanisms. The support for revisitation is normally through a set of functional built-in icons e.g. History, Back, Forward and Bookmarks. Nevertheless, for returning web users, they normally find it is easier and faster to re-launch an online search again, rather than spending time to find a particular web site from their personal bookmark and history records. Tauscher and Greenberg (1997) showed that revisiting web pages forms up to 58% of the recurrence rate of web browsing. Cockburn and McKenzie (2001) also stated that 81% of web pages have been previously visited by the user. According to Obendorf et al. (2007), revisitation can be divided into four classifications based on time: short-term (72.6% revisits within an hour), medium-term (12% revisits within a day and 7.8% revisits within a week), and long-term (7.6% revisits longer than a week)

Visual collaging of music in a digital library

This article explores the role visual browsing can play within a digital music library. The context to the work is provided through a review of related techniques drawn from the fields of digital libraries and human computer interaction. Implemented within the open source digital library toolkit Greenstone, a prototype system is described that combines images located through textual metadata with a visualisation technique known as collaging to provide a leisurely, undirected interaction with a music collection. Emphasis in the article is given to the augmentations of the basic technique to work in the musical domain

Children’s information retrieval: beyond examining search strategies and interfaces

The study of children’s information retrieval is still for the greater part untouched territory. Meanwhile, children can become lost in the digital information world, because they are confronted with search interfaces, both designed by and for adults. Most current research on children’s information retrieval focuses on examining children’s search performance on existing search interfaces to determine what kind of interfaces are suitable for children’s search behaviour. However, to discover the true nature of children’s search behaviour, we state that research has to go beyond examining search strategies used with existing search interfaces by examining children’s cognitive processes during information-seeking. A paradigm of children’s information retrieval should provide an overview of all the components beyond search interfaces and search strategies that are part of children’s information retrieval process. Better understanding of the nature of children’s search behaviour can help adults design interfaces and information retrieval systems that both support children’s natural search strategies and help them find their way in the digital information world

Interactive searching and browsing of video archives: using text and using image matching

Over the last number of decades much research work has been done in the general area of video and audio analysis. Initially the applications driving this included capturing video in digital form and then being able to store, transmit and render it, which involved a large effort to develop compression and encoding standards. The technology needed to do all this is now easily available and cheap, with applications of digital video processing now commonplace, ranging from CCTV (Closed Circuit TV) for security, to home capture of broadcast TV on home DVRs for personal viewing. One consequence of the development in technology for creating, storing and distributing digital video is that there has been a huge increase in the volume of digital video, and this in turn has created a need for techniques to allow effective management of this video, and by that we mean content management. In the BBC, for example, the archives department receives approximately 500,000 queries per year and has over 350,000 hours of content in its library. Having huge archives of video information is hardly any benefit if we have no effective means of being able to locate video clips which are of relevance to whatever our information needs may be. In this chapter we report our work on developing two specific retrieval and browsing tools for digital video information. Both of these are based on an analysis of the captured video for the purpose of automatically structuring into shots or higher level semantic units like TV news stories. Some also include analysis of the video for the automatic detection of features such as the presence or absence of faces. Both include some elements of searching, where a user specifies a query or information need, and browsing, where a user is allowed to browse through sets of retrieved video shots. We support the presentation of these tools with illustrations of actual video retrieval systems developed and working on hundreds of hours of video content

Designing a Semantically Rich Visual Iinterface for Cultural Digital Libraries Using the UNESCO Multilingual Thesaurus

This paper reports on the design of a visual user interface for the UNESCO digital portal. The interface makes use of the UNESCO multilingual thesaurus to provide visualized views of terms and their relationships and the way in which spaces associated with the thesaurus, the query and the results can be integrated into a single user interface.\u

Investigation of behavior and perception of digital library users: A cognitive style perspective

Cognitive style is an influential factor in users’ information seeking. The study presented in this paper examines how users’ cognitive styles affect their behavior and perception in digital libraries. Fifty participants took part in this study. Two dimensions of cognitive styles were considered: (a) Field Dependence/Independence; (2) Verbalizer/Imager. The results showed that Intermediate users and Verbalizers have not only more positive perception, but they also complete the tasks in effective ways. Implications for the design of personalized digital libraries are also discussed

Designing a semantically rich visual interface for cultural digital libraries using the UNEsCO multilingual thesaurus

This paper reports on the design of a visual user interface for the UNESCO digital portal. The interface makes use of the UNESCO multilingual thesaurus to provide visualized views of terms and their relationships and the way in which spaces associated with the thesaurus, the query and the results can be integrated into a single user interface