Video processing for panoramic streaming using HEVC and its scalable extensions

Panoramic streaming is a particular way of video streaming where an arbitrary Region-of-Interest (RoI) is transmitted from a high-spatial resolution video, i.e. a video covering a very “wide-angle” (much larger than the human field-of-view – e.g. 360°). Some transport schemes for panoramic video delivery have been proposed and demonstrated within the past decade, which allow users to navigate interactively within the high-resolution videos. With the recent advances of head mounted displays, consumers may soon have immersive and sufficiently convenient end devices at reach, which could lead to an increasing demand for panoramic video experiences. The solution proposed within this paper is built upon tile-based panoramic streaming, where users receive a set of tiles that match their RoI, and consists in a low-complexity compressed domain video processing technique for using H.265/HEVC and its scalable extensions (H.265/SHVC and H.265/MV-HEVC). The proposed technique generates a single video bitstream out of the selected tiles so that a single hardware decoder can be used. It overcomes the scalability issue of previous solutions not using tiles and the battery consumption issue inherent of tile-based panorama streaming, where multiple parallel software decoders are used. In addition, the described technique is capable of reducing peak streaming bitrate during changes of the RoI, which is crucial for allowing a truly immersive and low latency video experience. Besides, it makes it possible to use Open GOP structures without incurring any playback interruption at switching events, which provides a better compression efficiency compared to closed GOP structures

Immersion on the Edge: A Cooperative Framework for Mobile Immersive Computing

Immersive computing (IC) technologies such as virtual reality and augmented reality are gaining tremendous popularity. In this poster, we present CoIC, a Cooperative framework for mobile Immersive Computing. The design of CoIC is based on a key insight that IC tasks among different applications or users might be similar or redundant. CoIC enhances the performance of mobile IC applications by caching and sharing computation-intensive IC results on the edge. Our preliminary evaluation results on an AR application show that CoIC can reduce the recognition and rendering latency by up to 52.28% and 75.86% respectively on current mobile devices.Comment: This poster has been accepted by the SIGCOMM in June 201

Virtual viewpoint three-dimensional panorama

Conventional panoramic images are known to provide for an enhanced field of view in which the scene always has a fixed appearance. The idea presented in this paper focuses on the use of the concept of virtual viewpoint creation to generate different panoramic images of the same scene with three-dimensional component. Three-dimensional effect in a resultant panorama is realized by superimposing a stereo-pair of panoramic images

Cracks in the Glass: The Emergence of a New Image Typology from the Spatio-temporal Schisms of the 'Filmic' Virtual Reality Panorama

Virtual Reality Panoramas have fascinated me for some time; their interactive nature affording a spectatorial engagement not evident within other forms of painting or digital imagery. This interactivity is not generally linear as is evident in animation or film, nor is the engagement with the image reduced to the physical or visual border of the image, as its limit is never visible to the viewer in its entirety. Further, the time taken to interact and navigate across the Virtual Reality panorama’s surface is not reflected or recorded within the observed image. The procedural construction of the Virtual Reality panorama creates an a-temporal image event that denies the durée of its own index and creation. This is particularly evident in the cinematic experiments conducted by Jeffrey Shaw in the 1990s that ‘spatialised’ time and image through the fusion of the formal typology of the Panorama together with the cinematic moving-image, creating a new kind of image technology. The incorporation of the space enclosed by the panorama’s drum, into the conception and execution of the cinematic event, reveals an interesting conceptual paradox. Space and time infinitely and autonomously repeat upon each other as the linear trajectory of the singular cinematic shot is interrupted by a ‘time schism’ on the surface of the panorama. This paper explores what this conceptual paradox means to the evolution of emerging image-technologies and how Shaw’s ‘mixed-reality’ installation reveals a wholly new image typology that presents techniques and concepts though which to record, interrogate, and represent time and space in Architecture