Towards QoE-Driven Optimization of Multi-Dimensional Content Streaming

Whereas adaptive video streaming for 2D video is well established and frequently used in streaming services, adaptation for emerging higher-dimensional content, such as point clouds, is still a research issue. Moreover, how to optimize resource usage in streaming services that support multiple content types of different dimensions and levels of interactivity has so far not been sufficiently studied. Learning-based approaches aim to optimize the streaming experience according to user needs. They predict quality metrics and try to find system parameters maximizing them given the current network conditions. With this paper, we show how to approach content and network adaption driven by Quality of Experience (QoE) for multi-dimensional content. We describe components required to create a system adapting multiple streams of different content types simultaneously, identify research gaps and propose potential next steps

A Quick Survey of Volumetric Content Streaming Approaches

Volumetric content is an important enabler for a wide range of applications such as immersive real-time 3D communications and virtual reality content viewing with interactive parallax. While nowadays there is more and more hardware that captures and presents 3D representations of the world, streaming these representations, known as volumetric content, is a key problem to be addressed. Major challenges are related to the transfer of large amounts of unstructured 3D data over bandwidth-limited networks, instant response to users’ behavior, i.e. latency compensation, as well as computational complexity at both the server and client devices. To provide an overview of studies conducted in the field of volumetric content streaming, we research relevant literature, summarize different streaming schemes related to this focus. This paper provides a discussion of the challenges of volumetric content streaming, and an overview of the representative volumetric content streaming approaches proposed in the literature to date. Future directions and areas requiring further research are also discussed

Grand Challenge: 3-D Urban Objects Detection and Classification From Point Clouds

In this paper, we present our approach to solve the DEBS Grand challenge 2019 which consists of classifying urban objects in different scenes that originate from a LiDAR sensor. In general, at any point in time, LiDAR data can be considered as a point cloud where a reliable feature extractor and a classification model are required to be able to recognize 3-D objects in such scenes. Herein, we propose and describe an implementation of a 3-D point cloud object detection and classification system based on a 3-D global feature called Ensemble of Shape Functions (ESF) and a random forest object classifie

Content-Aware Adaptive Point Cloud Delivery

Point clouds are an important enabler for a wide range of applications in various domains, including autonomous vehicles and virtual reality applications. Hence, the practical applicability of point clouds is gaining increasing importance and presenting new challenges for communication systems where large amounts of data need to be shared with low latency. Point cloud content can be very large, especially when multiple objects are involved in the scene. Major challenges of point clouds delivery are related to streaming in bandwidth-constrained networks and to resource-constrained devices. In this work, we are exploiting object-related knowledge, i.e., content-driven metrics, to improve the adaptability and efficiency of point clouds transmission. This study proposes applying a 3D point cloud semantic segmentation deep neural network and using object related knowledge to assess the importance of each object in the scene. Using this information, we can semantically adapt the bit rate and utilize the available bandwidth more efficiently. The experimental results conducted on a real-world dataset showed that we can significantly reduce the requirement for multiple object point cloud transmission with limited quality degradation compared to the baseline without modifications

Modeling Quality of Experience for Compressed Point Cloud Sequences based on a Subjective Study

There is growing interest in point cloud content due to its central role in the creation and provision of interactive and immersive user experiences for extended reality applications. However, it is impractical to stream uncompressed point cloud sequences over communication networks to end systems because of their high throughput and low latency requirements. Several novel compression methods have been developed for efficient storage and adaptive delivery of point cloud content. However, these methods primarily focus on data metrics and neglect the influence on the actual Quality of Experience (QoE). In this paper, we conduct a user study with 102 participants to analyze the QoE of point cloud sequences and develop a QoE model that can enhance the quality of point cloud content distribution under dynamic network conditions. Our analysis is based on user opinions regarding two representative point cloud sequences, three different frame rates, three viewing distances, and two state-of-the-art point cloud compression libraries, Draco and V-PCC. The results indicate that the proposed models can accurately predict the users' quality perception, with frame rate being the most dominant QoE factor

Towards QoE-Driven Optimization of Multi-Dimensional Content Streaming

Whereas adaptive video streaming for 2D video is well established and frequently used in streaming services, adaptation for emerging higher-dimensional content, such as point clouds, is still a research issue. Moreover, how to optimize resource usage in streaming services that support multiple content types of different dimensions and levels of interactivity has so far not been sufficiently studied. Learning-based approaches aim to optimize the streaming experience according to user needs. They predict quality metrics and try to find system parameters maximizing them given the current network conditions. With this paper, we show how to approach content and network adaption driven by Quality of Experience (QoE) for multi-dimensional content. We describe components required to create a system adapting multiple streams of different content types simultaneously, identify research gaps and propose potential next steps

Towards QoE-Driven Optimization of Multi-Dimensional Content Streaming

Whereas adaptive video streaming for 2D video is well established and frequently used in streaming services, adaptation for emerging higher-dimensional content, such as point clouds, is still a research issue. Moreover, how to optimize resource usage in streaming services that support multiple content types of different dimensions and levels of interactivity has so far not been sufficiently studied. Learning-based approaches aim to optimize the streaming experience according to user needs. They predict quality metrics and try to find system parameters maximizing them given the current network conditions. With this paper, we show how to approach content and network adaption driven by Quality of Experience (QoE) for multi-dimensional content. We describe components required to create a system adapting multiple streams of different content types simultaneously, identify research gaps and propose potential next steps