Efficient Implementation and Processing of a Real-Time Panorama Video Pipeline

Abstract—High resolution, wide field of view video generated from multiple camera feeds has many use cases. However, processing the different steps of a panorama video pipeline in real-time is challenging due to the high data rates and the stringent requirements of timeliness. We use panorama video in a sport analysis system where video events must be generated in real-time. In this respect, we present a system for real-time panorama video generation from an array of low-cost CCD HD video cameras. We describe how we have implemented different components and evaluated alternatives. We also present performance results with and without co-processors like graphics processing units (GPUs), and we evaluate each individual component and show how the entire pipeline is able to run in real-time on commodity hardware. I

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Efficient implementation and processing of a real-time panorama video pipeline with emphasis on background subtraction

The Bagadus system has been introduced as an automated soccer analysis tool, and consists of an analysis subsystem, tracking subsystem and video subsystem. By automating the integration of these subsystems, Bagadus allows for simplified soccer analysis, with the goal of improving athletes' performance. The system is currently installed at Alfheim stadium in Tromsø, Norway. A part of the video subsystem is the generation of panorama videos from four HD cameras. However, the pipeline for panorama video generation in the first version of the system did not manage to do this online and in real-time. In this thesis, we present how to build an improved panorama stitcher pipeline that is able to stitch video from four HD cameras into a panorama video online and in real-time. We describe in detail the architecture and modules of this pipeline, and analyze the performance, where we demonstrate real-time, live capture, processing and storage of four individual camera feeds and generation of a panorama video on a single machine. In addition, we focus on how background subtraction can be used to improve the pipeline. As part of this, we discuss how we can utilize player position data to improve the background subtraction process, and also discuss in detail how to optimize the background subtraction process on CPU and GPU

Efficient implementation and processing of a real-time panorama video pipeline with emphasis on color correction

Today, several soccer teams make use of the capabilities provided by analysis systems to further improve the overall team and player performance. As more and more analysis systems are developed, using these types of tools have become common and the advantages these systems provides can be the difference between winning or loosing a match. To offer an analysis system that can capture team data, annotate important events and deliver real-time video, we present the idea of Bagadus. With all of the these goals fulfilled, it is able to provide all the capabilities needed for team development. The system combines player tracking sensors, a easy-to-use annotation system and cameras to capture video of the games. The goal of Bagadus is to be fully automatic and real-time, but currently there are several issues that needs to be solved before one can take full advantage of the system. In this thesis, we present an improved version of the previously existing Bagdus video capture component. By designing a pipeline for panorama creation, we are able to deliver a panorama video in real-time consisting of frames from four individual cameras. We will also look further into the visual output of the pipeline and how we can improve the image quality by applying a method for color correction

Efficient implementation and processing of a real-time panorama video pipeline with emphasis on dynamic stitching

The Bagadus system has been introduced as an automated tool for soccer analysis, and it is built up by an analysis subsystem, tracking subsystem and video subsystem. Bagadus allows for simplified soccer analysis, with the goal of improving athletes' performance, by automating the integration of these subsystems. The system is currently installed at Alfheim stadium in Tromsø, Norway. An important part of the video subsystem is the creation of panorama videos from four HD cameras. However, the stitching pipeline for panorama video generation in the first version of the system did not manage to do this in real-time. In this thesis, we present how to build an improved panorama stitcher pipeline that is able to stitch video from four HD cameras into a panorama video in real-time. We will detail the architecture and modules of this pipeline, and analyze the performance. In addition we will focus on the stitching component, and how that can improve the overall visual quality of the output panorama