The Video Browser Showdown: a live evaluation of interactive video search tools

The Video Browser Showdown evaluates the performance of exploratory video search tools on a common data set in a common environment and in presence of the audience. The main goal of this competition is to enable researchers in the field of interactive video search to directly compare their tools at work. In this paper, we present results from the second Video Browser Showdown (VBS2013) and describe and evaluate the tools of all participating teams in detail. The evaluation results give insights on how exploratory video search tools are used and how they perform in direct comparison. Moreover, we compare the achieved performance to results from another user study where 16 participants employed a standard video player to complete the same tasks as performed in VBS2013. This comparison shows that the sophisticated tools enable better performance in general, but for some tasks common video players provide similar performance and could even outperform the expert tools. Our results highlight the need for further improvement of professional tools for interactive search in videos

Visual access to lifelog data in a virtual environment

Continuous image capture via a wearable camera is currently one of the most popular methods to establish a comprehensive record of the entirety of an indi- vidual’s life experience, referred to in the research community as a lifelog. These vast image corpora are further enriched by content analysis and combined with additional data such as biometrics to generate as extensive a record of a person’s life as possible. However, interfacing with such datasets remains an active area of research, and despite the advent of new technology and a plethora of com- peting mediums for processing digital information, there has been little focus on newly emerging platforms such as virtual reality. We hypothesise that the increase in immersion, accessible spatial dimensions, and more, could provide significant benefits in the lifelogging domain over more conventional media. In this work, we motivate virtual reality as a viable method of lifelog exploration by performing an in-depth analysis using a novel application prototype built for the HTC Vive. This research also includes the development of a governing design framework for lifelog applications which supported the development of our prototype but is also intended to support the development of future such lifelog systems

Temporal multimodal video and lifelog retrieval

The past decades have seen exponential growth of both consumption and production of data, with multimedia such as images and videos contributing significantly to said growth. The widespread proliferation of smartphones has provided everyday users with the ability to consume and produce such content easily. As the complexity and diversity of multimedia data has grown, so has the need for more complex retrieval models which address the information needs of users. Finding relevant multimedia content is central in many scenarios, from internet search engines and medical retrieval to querying one's personal multimedia archive, also called lifelog. Traditional retrieval models have often focused on queries targeting small units of retrieval, yet users usually remember temporal context and expect results to include this. However, there is little research into enabling these information needs in interactive multimedia retrieval. In this thesis, we aim to close this research gap by making several contributions to multimedia retrieval with a focus on two scenarios, namely video and lifelog retrieval. We provide a retrieval model for complex information needs with temporal components, including a data model for multimedia retrieval, a query model for complex information needs, and a modular and adaptable query execution model which includes novel algorithms for result fusion. The concepts and models are implemented in vitrivr, an open-source multimodal multimedia retrieval system, which covers all aspects from extraction to query formulation and browsing. vitrivr has proven its usefulness in evaluation campaigns and is now used in two large-scale interdisciplinary research projects. We show the feasibility and effectiveness of our contributions in two ways: firstly, through results from user-centric evaluations which pit different user-system combinations against one another. Secondly, we perform a system-centric evaluation by creating a new dataset for temporal information needs in video and lifelog retrieval with which we quantitatively evaluate our models. The results show significant benefits for systems that enable users to specify more complex information needs with temporal components. Participation in interactive retrieval evaluation campaigns over multiple years provides insight into possible future developments and challenges of such campaigns

Visualising multiple overlapping classification hierarchies.

The revision or reorganisation of hierarchical data sets can result in many possible hierarchical classifications composed of the same or overlapping data sets existing in parallel with each other. These data sets are difficult for people to handle and conceptualise, as they tryto reconcile the different perspectives and structures that such data represents. One area where this situation occurs is the study of botanical taxonomy, essentially the classification and naming of plants. Revisions, new discoveries and new dimensions for classifying plants lead to a proliferation of classifications over the same set of plant data. Taxonomists would like a method of exploring these multiple overlapping hierarchies for interesting information, correlations, or anomalies. The application and extension of Information Visualisation (IV) techniques, the graphical display of abstract information, is put forward as a solution to this problem. Displaying the multiple classification hierarchies in a visually appealing manner along with powerful interaction mechanisms for examination and exploration of the data allows taxonomists to unearth previously hidden information. This visualisation gives detail that previous visualisations and statistical overviews cannot offer. This thesis work has extended previous IV work in several respects to achieve this goal. Compact, yet full and unambiguous, hierarchy visualisations have been developed. Linking and brushing techniques have been extended to work on a higher class of structure, namely overlapping trees and hierarchies. Focus and context techniques have been pushed to achieve new effects across the visually distinct representations of these multiple hierarchies. Other data types, such as multidimensional data and large cluster hierarchies have also been displayed using the final version of the visualisation