The Físchlár digital video recording, analysis, and browsing system

In digital video indexing research area an important technique is called shot boundary detection which automatically segments long video material into camera shots using content-based analysis of video. We have been working on developing various shot boundary detection and representative frame selection techniques to automatically index encoded video stream and provide the end users with video browsing/navigation feature. In this paper we describe a demonstrator digital video system that allows the user to record a TV broadcast programme to MPEG-1 file format and to easily browse and playback the file content online. The system incorporates the shot boundary detection and representative frame selection techniques we have developed and has become a full-featured digital video system that not only demonstrates any further techniques we will develop, but also obtains users’ video browsing behaviour. At the moment the system has a real-user base of about a hundred people and we are closely monitoring how they use the video browsing/navigation feature which the system provides

FoP: Never-Ending Learner for Multimedia Knowledge Extraction

International audience—In this paper we present our system Faces of Politics (henceforth FoP), that is able to continuously learn multimedia knowledge of Web multimedia resources about the presence of person(s) in a pictures and to leverage this knowledge to the Linked Open Data cloud (LOD-cloud). FoP promotes both scalability of the data lift process for this domain and a structured knowledge representation for complex queries. The system was bootstraped using Freebase data about politicians and their pictures, and we show that the model provides a good generalization with an error rate below 7%. Meantime, FoP not only relates a person to a multimedia resource, but it also detects and publishes metadata on the position of the person in the picture. Moreover, it supports the presence of several persons in the picture. At this step, FoP is also giving data in return to the LoD cloud that fed him in the first place: it leverages Linked Data on people recognized in these pictures, and on which rectangle area. This allows fine-grained queries like creating a curation of documents in which a person is depicted relatively to another for instance. On a technical point-of-view, we also provide a Website for browsing FoP knowledge base as Web users, and we also offer a public SPARQL endpoint for robots or other Web applications

Implementation and analysis of several keyframe-based browsing interfaces to digital video

In this paper we present a variety of browsing interfaces for digital video information. The six interfaces are implemented on top of Físchlár, an operational recording, indexing, browsing and playback system for broadcast TV programmes. In developing the six browsing interfaces, we have been informed by the various dimensions which can be used to distinguish one interface from another. For this we include layeredness (the number of “layers” of abstraction which can be used in browsing a programme), the provision or omission of temporal information (varying from full timestamp information to nothing at all on time) and visualisation of spatial vs. temporal aspects of the video. After introducing and defining these dimensions we then locate some common browsing interfaces from the literature in this 3-dimensional “space” and then we locate our own six interfaces in this same space. We then present an outline of the interfaces and include some user feedback

A class of structured P2P systems supporting browsing

Browsing is a way of finding documents in a large amount of data which is complementary to querying and which is particularly suitable for multimedia documents. Locating particular documents in a very large collection of multimedia documents such as the ones available in peer to peer networks is a difficult task. However, current peer to peer systems do not allow to do this by browsing. In this report, we show how one can build a peer to peer system supporting a kind of browsing. In our proposal, one must extend an existing distributed hash table system with a few features : handling partial hash-keys and providing appropriate routing mechanisms for these hash-keys. We give such an algorithm for the particular case of the Tapestry distributed hash table. This is a work in progress as no proper validation has been done yet.Comment: 14 page

Video Data Visualization System: Semantic Classification And Personalization

We present in this paper an intelligent video data visualization tool, based on semantic classification, for retrieving and exploring a large scale corpus of videos. Our work is based on semantic classification resulting from semantic analysis of video. The obtained classes will be projected in the visualization space. The graph is represented by nodes and edges, the nodes are the keyframes of video documents and the edges are the relation between documents and the classes of documents. Finally, we construct the user's profile, based on the interaction with the system, to render the system more adequate to its references.Comment: graphic

Multimedia search without visual analysis: the value of linguistic and contextual information

This paper addresses the focus of this special issue by analyzing the potential contribution of linguistic content and other non-image aspects to the processing of audiovisual data. It summarizes the various ways in which linguistic content analysis contributes to enhancing the semantic annotation of multimedia content, and, as a consequence, to improving the effectiveness of conceptual media access tools. A number of techniques are presented, including the time-alignment of textual resources, audio and speech processing, content reduction and reasoning tools, and the exploitation of surface features

Developing a MovieBrowser for supporting analysis and browsing of movie content

There is a growing awareness of the importance of system evaluation directly with end-users in realistic environments, and as a result some novel applications have been deployed to the real world and evaluated in trial contexts. While this is certainly a desirable trend to relate a technical system to a real user-oriented perspective, most of these efforts do not involve end-user participation right from the start of the development, but only after deploying it. In this paper we describe our research in designing, deploying and assessing the impact of a web-based tool that incorporates multimedia techniques to support movie analysis and browsing for students of film studies. From the very start and throughout the development we utilize methodologies from usability engineering in order to feed in end-user needs and thus tailoring the underlying technical system to those needs. Starting by capturing real users’ current practices and matching them to the available technical elements of the system, we deployed an initial version of our system to University classes for a semester during which we obtained an extensive amount of rich usage data. We describe the process and some of the findings from this trial

Interactive searching and browsing of video archives: using text and using image matching

Over the last number of decades much research work has been done in the general area of video and audio analysis. Initially the applications driving this included capturing video in digital form and then being able to store, transmit and render it, which involved a large effort to develop compression and encoding standards. The technology needed to do all this is now easily available and cheap, with applications of digital video processing now commonplace, ranging from CCTV (Closed Circuit TV) for security, to home capture of broadcast TV on home DVRs for personal viewing. One consequence of the development in technology for creating, storing and distributing digital video is that there has been a huge increase in the volume of digital video, and this in turn has created a need for techniques to allow effective management of this video, and by that we mean content management. In the BBC, for example, the archives department receives approximately 500,000 queries per year and has over 350,000 hours of content in its library. Having huge archives of video information is hardly any benefit if we have no effective means of being able to locate video clips which are of relevance to whatever our information needs may be. In this chapter we report our work on developing two specific retrieval and browsing tools for digital video information. Both of these are based on an analysis of the captured video for the purpose of automatically structuring into shots or higher level semantic units like TV news stories. Some also include analysis of the video for the automatic detection of features such as the presence or absence of faces. Both include some elements of searching, where a user specifies a query or information need, and browsing, where a user is allowed to browse through sets of retrieved video shots. We support the presentation of these tools with illustrations of actual video retrieval systems developed and working on hundreds of hours of video content