Foveated Video Streaming for Cloud Gaming

Good user experience with interactive cloud-based multimedia applications, such as cloud gaming and cloud-based VR, requires low end-to-end latency and large amounts of downstream network bandwidth at the same time. In this paper, we present a foveated video streaming system for cloud gaming. The system adapts video stream quality by adjusting the encoding parameters on the fly to match the player's gaze position. We conduct measurements with a prototype that we developed for a cloud gaming system in conjunction with eye tracker hardware. Evaluation results suggest that such foveated streaming can reduce bandwidth requirements by even more than 50% depending on parametrization of the foveated video coding and that it is feasible from the latency perspective.Comment: Submitted to: IEEE 19th International Workshop on Multimedia Signal Processin

Pruning Edge Research with Latency Shears

Edge computing has gained attention from both academia and industry by pursuing two significant challenges: 1) moving latency critical services closer to the users, 2) saving network bandwidth by aggregating large flows before sending them to the cloud. While the rationale appeared sound at its inception almost a decade ago, several current trends are impacting it. Clouds have spread geographically reducing end-user latency, mobile phones? computing capabilities are improving, and network bandwidth at the core keeps increasing. In this paper, we scrutinize edge computing, examining its outlook and future in the context of these trends. We perform extensive client-to-cloud measurements using RIPE Atlas, and show that latency reduction as motivation for edge is not as persuasive as once believed; for most applications the cloud is already 'close enough' for majority of the world's population. This implies that edge computing may only be applicable for certain application niches, as opposed to a general-purpose solution.Peer reviewe

Foveated Video Streaming for Cloud Gaming

Video gaming is generally a computationally intensive application and to provide a pleasant user experience specialized hardware like Graphic Processing Units may be required. Computational resources and power consumption are constraints which limit visually complex gaming on, for example, laptops, tablets and smart phones. Cloud gaming may be a possible approach towards providing a pleasant gaming experience on thin clients which have limited computational and energy resources. In a cloud gaming architecture, the game-play video is rendered and encoded in the cloud and streamed to a client where it is displayed. User inputs are captured at the client and streamed back to the server, where they are relayed to the game. High quality of experience requires the streamed video to be of high visual quality which translates to substantial downstream bandwidth requirements. The visual perception of the human eye is non-uniform, being maximum along the optical axis of the eye and dropping off rapidly away from it. This phenomenon, called foveation, makes the practice of encoding all areas of a video frame with the same resolution wasteful. In this thesis, foveated video streaming from a cloud gaming server to a cloud gaming client is investigated. A prototype cloud gaming system with foveated video streaming is implemented. The cloud gaming server of the prototype is configured to encode gameplay video in a foveated fashion based on gaze location data provided by the cloud gaming client. The effect of foveated encoding on the output bitrate of the streamed video is investigated. Measurements are performed using games from various genres and with different player points of view to explore changes in video bitrate with different parameters of foveation. Latencies involved in foveated video streaming for cloud gaming, including latency of the eye tracker used in the thesis, are also briefly discussed

When Internet of Things meets Metaverse: Convergence of Physical and Cyber Worlds

In recent years, the Internet of Things (IoT) is studied in the context of the Metaverse to provide users immersive cyber-virtual experiences in mixed reality environments. This survey introduces six typical IoT applications in the Metaverse, including collaborative healthcare, education, smart city, entertainment, real estate, and socialization. In the IoT-inspired Metaverse, we also comprehensively survey four pillar technologies that enable augmented reality (AR) and virtual reality (VR), namely, responsible artificial intelligence (AI), high-speed data communications, cost-effective mobile edge computing (MEC), and digital twins. According to the physical-world demands, we outline the current industrial efforts and seven key requirements for building the IoT-inspired Metaverse: immersion, variety, economy, civility, interactivity, authenticity, and independence. In addition, this survey describes the open issues in the IoT-inspired Metaverse, which need to be addressed to eventually achieve the convergence of physical and cyber worlds.info:eu-repo/semantics/publishedVersio

Dynamic Viewport-Adaptive Rendering in Distributed Interactive VR Streaming: Optimizing viewport resolution under latency and viewport orientation constraints

In streaming Virtual Reality to thin clients one of the main concerns is the massive bandwidth requirement of VR video. Additionally, streaming VR requires a low latency of less than 25ms to avoid cybersickness and provide a high Quality of Experience. Since a user is only viewing a portion of the VR content sphere at a time, researchers have leveraged this to increase the relative quality of the user viewport compared to peripheral areas. This way bandwidth can be saved, since the peripheral areas are streamed at a lower bitrate. In streaming $360\degree$ video this has resulted in the common strategy of tiling a video frame and delivering different quality tiles based on current available bandwidth and the user's viewport location. However, such an approach is not suitable for real-time Interactive VR streaming. Furthermore, streaming only the user's viewport results in the user observing unrendered or very low-quality areas at higher latency values. In order to provide a high viewport quality in Interactive VR, we propose the novel method of Dynamic Viewport-Adaptive Rendering. By rotating the frontal direction of the content sphere with the user gaze, we can dynamically render more or less of the peripheral area and thus increase the proportional resolution of the frontal direction in the video frame. We show that DVAR can successfully compensate for different system RTT values while offering a significantly higher viewport resolution than other implementations. We further discuss how DVAR can be easily extended by other optimization methods and discuss how we can incorporate head movement prediction to allow DVAR to optimally determine the amount of peripheral area to render, thus providing an optimal viewport resolution given the system constraints

Application of Quality of Experience in Networked Services: Review, Trend & Perspectives

Full text embargoed until 17.10.2019 (publisher's embargo period, 12 months

6G Enabled Smart Infrastructure for Sustainable Society: Opportunities, Challenges, and Research Roadmap

The 5G wireless communication network is currently faced with the challenge of limited data speed exacerbated by the proliferation of billions of data-intensive applications. To address this problem, researchers are developing cutting-edge technologies for the envisioned 6G wireless communication standards to satisfy the escalating wireless services demands. Though some of the candidate technologies in the 5G standards will apply to 6G wireless networks, key disruptive technologies that will guarantee the desired quality of physical experience to achieve ubiquitous wireless connectivity are expected in 6G. This article first provides a foundational background on the evolution of different wireless communication standards to have a proper insight into the vision and requirements of 6G. Second, we provide a panoramic view of the enabling technologies proposed to facilitate 6G and introduce emerging 6G applications such as multi-sensory–extended reality, digital replica, and more. Next, the technology-driven challenges, social, psychological, health and commercialization issues posed to actualizing 6G, and the probable solutions to tackle these challenges are discussed extensively. Additionally, we present new use cases of the 6G technology in agriculture, education, media and entertainment, logistics and transportation, and tourism. Furthermore, we discuss the multi-faceted communication capabilities of 6G that will contribute significantly to global sustainability and how 6G will bring about a dramatic change in the business arena. Finally, we highlight the research trends, open research issues, and key take-away lessons for future research exploration in 6G wireless communicatio