Video browsing interfaces and applications: a review

We present a comprehensive review of the state of the art in video browsing and retrieval systems, with special emphasis on interfaces and applications. There has been a significant increase in activity (e.g., storage, retrieval, and sharing) employing video data in the past decade, both for personal and professional use. The ever-growing amount of video content available for human consumption and the inherent characteristics of video data—which, if presented in its raw format, is rather unwieldy and costly—have become driving forces for the development of more effective solutions to present video contents and allow rich user interaction. As a result, there are many contemporary research efforts toward developing better video browsing solutions, which we summarize. We review more than 40 different video browsing and retrieval interfaces and classify them into three groups: applications that use video-player-like interaction, video retrieval applications, and browsing solutions based on video surrogates. For each category, we present a summary of existing work, highlight the technical aspects of each solution, and compare them against each other

A sensing platform for physiological and contextual feedback to tennis athletes

In this paper we describe our work on creating a multi-modal sensing platform for providing feedback to tennis coaches and players. The platform includes a fixed installation around a tennis court consisting of a video camera network and a localisation system as well as wearable sensing technology deployed to individual athletes. We describe the various components of this platform and explain how we can capture synchronised multi-modal sensor data streams for games or training sessions. We then describe the content-based retrieval system we are building to facilitate the development of novel coaching tools. We provide some examples of the queries that the system can support, where these queries are chosen to be suitably expressive so as to reflect a coach's complex information needs regarding tennis-related performance factors

Dynamic optimization of the quality of experience during mobile video watching

Mobile video consumption through streaming is becoming increasingly popular. The video parameters for an optimal quality are often automatically determined based on device and network conditions. Current mobile video services typically decide on these parameters before starting the video streaming and stick to these parameters during video playback. However in a mobile environment, conditions may change significantly during video playback. Therefore, this paper proposes a dynamic optimization of the quality taking into account real-time data regarding network, device, and user movement during video playback. The optimization method is able to change the video quality level during playback if changing conditions require this. Through a user test, the dynamic optimization is compared with a traditional, static, quality optimization method. The results showed that our optimization can improve the perceived playback and video quality, especially under varying network conditions

Flexi-WVSNP-DASH: A Wireless Video Sensor Network Platform for the Internet of Things

abstract: Video capture, storage, and distribution in wireless video sensor networks (WVSNs) critically depends on the resources of the nodes forming the sensor networks. In the era of big data, Internet of Things (IoT), and distributed demand and solutions, there is a need for multi-dimensional data to be part of the Sensor Network data that is easily accessible and consumable by humanity as well as machinery. Images and video are expected to become as ubiquitous as is the scalar data in traditional sensor networks. The inception of video-streaming over the Internet, heralded a relentless research for effective ways of distributing video in a scalable and cost effective way. There has been novel implementation attempts across several network layers. Due to the inherent complications of backward compatibility and need for standardization across network layers, there has been a refocused attention to address most of the video distribution over the application layer. As a result, a few video streaming solutions over the Hypertext Transfer Protocol (HTTP) have been proposed. Most notable are Apple’s HTTP Live Streaming (HLS) and the Motion Picture Experts Groups Dynamic Adaptive Streaming over HTTP (MPEG-DASH). These frameworks, do not address the typical and future WVSN use cases. A highly flexible Wireless Video Sensor Network Platform and compatible DASH (WVSNP-DASH) are introduced. The platform's goal is to usher video as a data element that can be integrated into traditional and non-Internet networks. A low cost, scalable node is built from the ground up to be fully compatible with the Internet of Things Machine to Machine (M2M) concept, as well as the ability to be easily re-targeted to new applications in a short time. Flexi-WVSNP design includes a multi-radio node, a middle-ware for sensor operation and communication, a cross platform client facing data retriever/player framework, scalable security as well as a cohesive but decoupled hardware and software design.Dissertation/ThesisDoctoral Dissertation Electrical Engineering 201

INTERFACE FOR SMALL-SCREEN MEDIA PLAYBACK CONTROL

A media player (e.g., a mobile phone, a tablet computer, a laptop computer, a television, etc.) may present a graphical user interface (GUI) that includes controls for interacting with a media file. The controls may include a play button, a pause button, and a progress bar with a marker designating the present point of playback (i.e., the current frame). The media player may enable user manipulation of the present point of playback, also referred to as media scrubbing, by moving the marker along the progress bar in response to user input dragging the marker along the progress bar. The GUI may be displayed on a screen of any size and shape (e.g., rectangular or non-rectangular). Displaying the GUI on a small or non-rectangular display, such as on a smart watch, may inhibit effective media scrubbing by making the progress bar too small for effective and/or accurate user interaction (e.g., smaller screens may result in a size of the progress bar becoming relatively small as compared to a size of a user’s finger). This disclosure provides techniques that enable effective and/or accurate media scrubbing on a small and/or non-rectangular display

Engaging Students through Video: Integrating Assessment and Instrumentation

CS50 is Harvard’s introductory course for majors and non-majors alike. For years, we have posted videos of the course’s lectures and sections online for the sake of review and distance education alike. But students’ experience with these videos has been historically passive. Students have been able to watch the course’s content on demand, rewinding and fast-forwarding at will, but they have not had means to engage interactively with the content or to check their understanding of material while watching videos. Furthermore, while we collected basic usage data (e.g., how many times a video was viewed), we lacked detailed analytics describing, for example, which portions of a video were commonly skipped or watched multiple times by students. To make videos more immersive and engaging for students, we developed CS50 Video, an open-source video player for desktop and mobile devices. CS50 Video allows instructors to integrate assessment questions to be answered by students at their own pace or at specific points in time directly into a video player. CS50 Video also allows students to search over video transcripts to find content easily as well as view videos at variable playback speeds (in order to make videos more accessible for ESL learners). Finally, CS50 Video integrates with third-party analytics solutions to allow instructors to view detailed usage statistics describing how students are interacting with videos (e.g., which videos or portions of videos are commonly watched or skipped over). We have deployed CS50 Video to students taking CS50 online and have obtained preliminary results. Because CS50 Video stores responses to questions server-side, we have been able to track students’ performance on in-video assessments. Thus far, we have observed that only 28% of students who watch online videos have engaged with assessment questions. Students who answer an assessment question incorrectly on their first attempt will often try again until reaching a correct answer, with 84.5% of correct answers reached in at most three attempts. We next plan to analyze the effects of in-video assessments on students’ mastery of material and introduce A/B-testing functionality for questions. We also plan to use students’ performance on assessments to understand the topics with which students struggle.Engineering and Applied Science

Analysis and design of a subtitling system for ambient intelligence environments

The development of ubiquitous applications for ambient intelligence environments needs to also take into account some usability and accessibility issues in order to ensure a proper user experience and to overcome the existing content access barriers. A proper access to video subtitles, for instance, is not always available due to the technical limitations of traditional video packaging, transmission and presentation. New Web standards enable more featured applications with better multi-platform definition, so they are suitable for building ubiquitous applications for ambient intelligence environments. This work presents a video subtitling system that enables the customization and adaptation of subtitles. The benefits of Web applications compared with device-specific native applications for building the solution as well as its current platform support are analyzed. Finally, three different application use cases are presented

Design of a player-plugin for metadata visualization and intelligent navigation

This project deals with the implementation of a metadata visualisation tool for the neural network developed by Ugiat Technologies. As it is a video processing software, the implemented tool is a player with extra features that allows to check if the metadata provided by the neural network is right. It has been done using web technologies for better portability, mainly in JavaScript. This way almost any device can use it, but it is created for desktop use, no matter the operating system, since it runs in a web browser, but the screen resolution should be the one of a laptop or a desktop computer, no mobile usage is covered by the design. For a better user experience, it allows voice commands to control some basic functions like playing, pausing and stopping the video as well as for enabling and disabling some of the extra features added to the video player. All the implementation is completely open source and it allows to continue adding functionalities. The product obtained is good not just for helping the development and tests of the processing software, but also for making demos of its performance

Proxy-based near real-time TV content transmission in mobility over 4G with MPEG-DASH transcoding on the cloud

[EN] This paper presents and evaluates a system that provides TV and radio services in mobility using 4G communications. The system has mainly two blocks, one on the cloud and another on the mobile vehicle. On the cloud, a DVB (Digital Video Broadcasting) receiver obtains the TV/radio signal and prepares the contents to be sent through 4G. Specifically, contents are transcoded and packetized using the DASH (Dynamic Adaptive Streaming over HTTP) standard. Vehicles in mobility use their 4G connectivity to receive the flows transmitted by the cloud. The key element of the system is an on-board proxy that manages the received flows and offers them to the final users in the vehicle. The proxy contains a buffer that helps reduce the number of interruptions caused by hand over effects and lack of coverage. The paper presents a comparison between a live transmission using 4G connecting the clients directly with the cloud server and a near real-time transmission based on an on-board proxy. Results prove that the use of the proxy reduces the number of interruptions considerably and, thus, improves the Quality of Experience of users at the expense of slightly increasing the delay.This work is supported by the Centro para el Desarrollo Tecnologico Industrial (CDTI) from the Government of Spain under the project "Plataforma avanzada de conectividad en movilidad" (CDTI IDI-20150126) and the project "Desarrollo de nueva plataforma de entretenimiento multimedia para entornos nauticos" (CDTI TIC-20170102).Arce Vila, P.; De Fez Lava, I.; Belda Ortega, R.; Guerri Cebollada, JC.; Ferrairó, S. (2019). Proxy-based near real-time TV content transmission in mobility over 4G with MPEG-DASH transcoding on the cloud. Multimedia Tools and Applications. 78(18):26399-26425. https://doi.org/10.1007/s11042-019-07840-6S2639926425781