508 research outputs found
Foveated Video Streaming for Cloud Gaming
Good user experience with interactive cloud-based multimedia applications,
such as cloud gaming and cloud-based VR, requires low end-to-end latency and
large amounts of downstream network bandwidth at the same time. In this paper,
we present a foveated video streaming system for cloud gaming. The system
adapts video stream quality by adjusting the encoding parameters on the fly to
match the player's gaze position. We conduct measurements with a prototype that
we developed for a cloud gaming system in conjunction with eye tracker
hardware. Evaluation results suggest that such foveated streaming can reduce
bandwidth requirements by even more than 50% depending on parametrization of
the foveated video coding and that it is feasible from the latency perspective.Comment: Submitted to: IEEE 19th International Workshop on Multimedia Signal
Processin
Foveated Video Streaming for Cloud Gaming
Video gaming is generally a computationally intensive application and to provide a pleasant user experience specialized hardware like Graphic Processing Units may be required. Computational resources and power consumption are constraints which limit visually complex gaming on, for example, laptops, tablets and smart phones. Cloud gaming may be a possible approach towards providing a pleasant gaming experience on thin clients which have limited computational and energy resources. In a cloud gaming architecture, the game-play video is rendered and encoded in the cloud and streamed to a client where it is displayed. User inputs are captured at the client and streamed back to the server, where they are relayed to the game. High quality of experience requires the streamed video to be of high visual quality which translates to substantial downstream bandwidth requirements. The visual perception of the human eye is non-uniform, being maximum along the optical axis of the eye and dropping off rapidly away from it. This phenomenon, called foveation, makes the practice of encoding all areas of a video frame with the same resolution wasteful.
In this thesis, foveated video streaming from a cloud gaming server to a cloud gaming client is investigated. A prototype cloud gaming system with foveated video streaming is implemented. The cloud gaming server of the prototype is configured to encode gameplay video in a foveated fashion based on gaze location data provided by the cloud gaming client. The effect of foveated encoding on the output bitrate of the streamed video is investigated. Measurements are performed using games from various genres and with different player points of view to explore changes in video bitrate with different parameters of foveation. Latencies involved in foveated video streaming for cloud gaming, including latency of the eye tracker used in the thesis, are also briefly discussed
From capturing to rendering : volumetric media delivery with six degrees of freedom
Technological improvements are rapidly advancing holographic-type content distribution. Significant research efforts have been made to meet the low latency and high bandwidth requirements set forward by interactive applications such as remote surgery and virtual reality. Recent research made six degrees of freedom (6DoF) for immersive media possible, where users may both move their head and change their position within a scene. In this article, we present the status and challenges of 6DoF applications based on volumetric media, focusing on the key aspects required to deliver such services. Furthermore, we present results from a subjective study to highlight relevant directions for future research
Low-latency Cloud-based Volumetric Video Streaming Using Head Motion Prediction
Volumetric video is an emerging key technology for immersive representation
of 3D spaces and objects. Rendering volumetric video requires lots of
computational power which is challenging especially for mobile devices. To
mitigate this, we developed a streaming system that renders a 2D view from the
volumetric video at a cloud server and streams a 2D video stream to the client.
However, such network-based processing increases the motion-to-photon (M2P)
latency due to the additional network and processing delays. In order to
compensate the added latency, prediction of the future user pose is necessary.
We developed a head motion prediction model and investigated its potential to
reduce the M2P latency for different look-ahead times. Our results show that
the presented model reduces the rendering errors caused by the M2P latency
compared to a baseline system in which no prediction is performed.Comment: 7 pages, 4 figure
Live HDR video streaming on commodity hardware
High Dynamic Range (HDR) video provides a step change in viewing experience, for example the ability to clearly see the soccer ball when it is kicked from the shadow of the stadium into sunshine. To achieve the full potential of HDR video, so-called true HDR, it is crucial that all the dynamic range that was captured is delivered to the display device and tone mapping is confined only to the display. Furthermore, to ensure widespread uptake of HDR imaging, it should be low cost and available on commodity hardware. This paper describes an end-to-end HDR pipeline for capturing, encoding and streaming high-definition HDR video in real-time using off-the-shelf components. All the lighting that is captured by HDR-enabled consumer cameras is delivered via the pipeline to any display, including HDR displays and even mobile devices with minimum latency. The system thus provides an integrated HDR video pipeline that includes everything from capture to post-production, archival and storage, compression, transmission, and display
Network streaming and compression for mixed reality tele-immersion
Bulterman, D.C.A. [Promotor]Cesar, P.S. [Copromotor
Performance and Complexity Co-Evaluations of MPEG4-ALS Compression Standard for Low-Latency Music Compression
In this thesis compression ratio and latency of different classical audio music tracks are analyzed with various encoder options of MPEG4ALS. Different tracks of audio music tracks are tested with MPEG4-ALS coder with different options to find the optimum values for various parameters to obtain maximum compression ratio with minimum CPU time (encoder and decoder time). Optimum frame length for which the compression ratio saturates for music audio is found out by analyzing the results when different classical music tracks are experimented with various frame lengths. Also music tracks with varying sampling rate are tested and the compression ratio and latency relationship with sampling rate are analyzed and plotted. It is found that the compression gain rate was higher when the codec complexity is less, and joint channel correlation and long term correlations are not significant and latency trade off make the more complex codec options unsuitable for applications where latency is critical. When the two entropy coding options, Rice code and BGMC (Block Gilbert-Moore Codes) are applied on various classical music tracks, it was obvious that the Rice code is more suitable for low-latency applications compared to the more complex BGMC coding, as BGMC improved compression performance with the expense of latency, making it unsuitable in real-time applications
- …