23 research outputs found
Foveated Video Streaming for Cloud Gaming
Good user experience with interactive cloud-based multimedia applications,
such as cloud gaming and cloud-based VR, requires low end-to-end latency and
large amounts of downstream network bandwidth at the same time. In this paper,
we present a foveated video streaming system for cloud gaming. The system
adapts video stream quality by adjusting the encoding parameters on the fly to
match the player's gaze position. We conduct measurements with a prototype that
we developed for a cloud gaming system in conjunction with eye tracker
hardware. Evaluation results suggest that such foveated streaming can reduce
bandwidth requirements by even more than 50% depending on parametrization of
the foveated video coding and that it is feasible from the latency perspective.Comment: Submitted to: IEEE 19th International Workshop on Multimedia Signal
Processin
Foveated Video Streaming for Cloud Gaming
Video gaming is generally a computationally intensive application and to provide a pleasant user experience specialized hardware like Graphic Processing Units may be required. Computational resources and power consumption are constraints which limit visually complex gaming on, for example, laptops, tablets and smart phones. Cloud gaming may be a possible approach towards providing a pleasant gaming experience on thin clients which have limited computational and energy resources. In a cloud gaming architecture, the game-play video is rendered and encoded in the cloud and streamed to a client where it is displayed. User inputs are captured at the client and streamed back to the server, where they are relayed to the game. High quality of experience requires the streamed video to be of high visual quality which translates to substantial downstream bandwidth requirements. The visual perception of the human eye is non-uniform, being maximum along the optical axis of the eye and dropping off rapidly away from it. This phenomenon, called foveation, makes the practice of encoding all areas of a video frame with the same resolution wasteful.
In this thesis, foveated video streaming from a cloud gaming server to a cloud gaming client is investigated. A prototype cloud gaming system with foveated video streaming is implemented. The cloud gaming server of the prototype is configured to encode gameplay video in a foveated fashion based on gaze location data provided by the cloud gaming client. The effect of foveated encoding on the output bitrate of the streamed video is investigated. Measurements are performed using games from various genres and with different player points of view to explore changes in video bitrate with different parameters of foveation. Latencies involved in foveated video streaming for cloud gaming, including latency of the eye tracker used in the thesis, are also briefly discussed
FOVQA: Blind Foveated Video Quality Assessment
Previous blind or No Reference (NR) video quality assessment (VQA) models
largely rely on features drawn from natural scene statistics (NSS), but under
the assumption that the image statistics are stationary in the spatial domain.
Several of these models are quite successful on standard pictures. However, in
Virtual Reality (VR) applications, foveated video compression is regaining
attention, and the concept of space-variant quality assessment is of interest,
given the availability of increasingly high spatial and temporal resolution
contents and practical ways of measuring gaze direction. Distortions from
foveated video compression increase with increased eccentricity, implying that
the natural scene statistics are space-variant. Towards advancing the
development of foveated compression / streaming algorithms, we have devised a
no-reference (NR) foveated video quality assessment model, called FOVQA, which
is based on new models of space-variant natural scene statistics (NSS) and
natural video statistics (NVS). Specifically, we deploy a space-variant
generalized Gaussian distribution (SV-GGD) model and a space-variant
asynchronous generalized Gaussian distribution (SV-AGGD) model of mean
subtracted contrast normalized (MSCN) coefficients and products of neighboring
MSCN coefficients, respectively. We devise a foveated video quality predictor
that extracts radial basis features, and other features that capture
perceptually annoying rapid quality fall-offs. We find that FOVQA achieves
state-of-the-art (SOTA) performance on the new 2D LIVE-FBT-FCVR database, as
compared with other leading FIQA / VQA models. we have made our implementation
of FOVQA available at: http://live.ece.utexas.edu/research/Quality/FOVQA.zip
Video Caching, Analytics and Delivery at the Wireless Edge: A Survey and Future Directions
Future wireless networks will provide high bandwidth, low-latency, and ultra-reliable Internet connectivity to meet the requirements of different applications, ranging from mobile broadband to the Internet of Things. To this aim, mobile edge caching, computing, and communication (edge-C3) have emerged to bring network resources (i.e., bandwidth, storage, and computing) closer to end users. Edge-C3 allows improving the network resource utilization as well as the quality of experience (QoE) of end users. Recently, several video-oriented mobile applications (e.g., live content sharing, gaming, and augmented reality) have leveraged edge-C3 in diverse scenarios involving video streaming in both the downlink and the uplink. Hence, a large number of recent works have studied the implications of video analysis and streaming through edge-C3. This article presents an in-depth survey on video edge-C3 challenges and state-of-the-art solutions in next-generation wireless and mobile networks. Specifically, it includes: a tutorial on video streaming in mobile networks (e.g., video encoding and adaptive bitrate streaming); an overview of mobile network architectures, enabling technologies, and applications for video edge-C3; video edge computing and analytics in uplink scenarios (e.g., architectures, analytics, and applications); and video edge caching, computing and communication methods in downlink scenarios (e.g., collaborative, popularity-based, and context-aware). A new taxonomy for video edge-C3 is proposed and the major contributions of recent studies are first highlighted and then systematically compared. Finally, several open problems and key challenges for future research are outlined