Measuring user Quality of Experience in social VR systems

Virtual Reality (VR) is a computer-generated experience that can simulate physical presence in real or imagined environments [7]. A social VR system is an application that allows multiple users to join a collaborative Virtual Environment (VE), such as a computer-generated 3D scene or a 360-degree natural scene captured by an omnidirectional camera, and communicate with each other, usually by means of visual and audio cues. Each user is represented in the VE as a computer-generated avatar [3] or, in recently proposed systems, with a virtual representation based on live captures [1]. Depending on the system, the user’ virtual representation can also interact with the virtual environment, for example by manipulating virtual objects, controlling the appearance of the VE, or controlling the playout of additional media in the VE. The interest for social Virtual Reality (VR) systems dates back to the late 90s [4, 8] but has recently increased [2, 5, 6] due to the availability of affordable head-mounted displays on the consumer market and to the appearance of new applications, such as Facebook Spaces, YouTube VR, Hulu VR, which explicitly aim at including social features in existing VR platforms for multimedia delivery. In this talk, we will address the problem of measuring user Quality of Experience (QoE) in social VR systems. We will review the studies that have analysed how different features of a social VR system design, such as avatar appearance and behavioural realism, can affect user’s experience, and propose a comparison of the objective and subjective measures used in the literature to quantify user QoE in social VR. Finally, we will discuss the use case of watching movies together in VR and present the results of one of our recent studies focusing on this scenario, designed and performed in the framework of the European project VRTogether (http://vrtogether.eu). Particularly, we show an analysis of correlation between the objective and subjective measurements collected during our study, to provide guidelines toward the design of a unified methodology to monitor and quantify users’ QoE in social VR systems. The open questions to be addressed in the future in order to achieve such goal are also discussed

Complexity measurement and characterization of 360-degree content

The appropriate characterization of the test material, used for subjective evaluation tests and for benchmarking image and video processing algorithms and quality metrics, can be crucial in order to perform comparative studies that provide useful insights. This paper focuses on the characterisation of 360-degree images. We discuss why it is important to take into account the geometry of the signal and the interactive nature of 360-degree content navigation, for a perceptual characterization of these signals. Particularly, we show that the computation of classical indicators of spatial complexity, commonly used for 2D images, might lead to different conclusions depending on the geometrical domain use

Complexity measurement and characterization of 360-degree content

The appropriate characterization of the test material, used for subjective evaluation tests and for benchmarking image and video processing algorithms and quality metrics, can be crucial in order to perform comparative studies that provide useful insights. This paper focuses on the characterisation of 360-degree images. We discuss why it is important to take into account the geometry of the signal and the interactive nature of 360-degree content navigation, for a perceptual characterization of these signals. Particularly, we show that the computation of classical indicators of spatial complexity, commonly used for 2D images, might lead to different conclusions depending on the geometrical domain use

Graph-Based Detection of Seams In 360-Degree Images

In this paper, we propose an algorithm to detect a specific kind of distortions, referred to as seams, which commonly oc- cur when a 360-degree image is represented in planar domain by projecting the sphere to a polyhedron, e.g, via the Cube Map (CM) projection, and undergoes lossy compression. The proposed algorithm exploits a graph-based representation to account for the actual sampling density of the 360-degree sig- nal in the native spherical domain. The CM image is con- sidered as a signal lying on a graph defined on the spherical surface. The spectra of the processed and the original sig- nals, computed by applying the Graph Fourier Transform, are compared to detect the seams. To test our method a dataset of compressed CM 360-degree images, annotated by experts, has been created. The performance of the proposed algorithm is compared to those achieved by baseline metrics, as well as to the same approach based on spectral comparison but ignor- ing the spherical nature of the signal. The experimental results show that the proposed method has the best performance and can successfully detect up to approximately 90% of visible seams on our dataset

Rate distortion optimized graph partitioning for omnidirectional image coding

International audienceOmnidirectional images are spherical signals captured by cameras with 360-degree field of view. In order to be compressed using existing encoders, these signals are mapped to planar domain. A commonly used planar representation is the equirectangular one, which corresponds to a non uniform sampling pattern on the spherical surface. This particularity is not explored in traditional image compression schemes, which treat the input signal as a classical perspective image. In this work, we build a graph-based coder adapted to the spherical surface. We build a graph directly on the sphere. Then, to have computationally feasible graph transforms, we propose a rate-distortion optimized graph partitioning algorithm to achieve an effective trade-off between the distortion of the reconstructed signals, the smoothness of the signal on each subgraph, and the cost of coding the graph partitioning description. Experimental results demonstrate that our method outperforms JPEG coding of planar equirectangular images

Temporal Interpolation of Dynamic Digital Humans using Convolutional Neural Networks

In recent years, there has been an increased interest in point cloud representation for visualizing digital humans in cross reality. However, due to their voluminous size, point clouds require high bandwidth to be transmitted. In this paper, we propose a temporal interpolation architecture capable of increasing the temporal resolution of dynamic digital humans, represented using point clouds. With this technique, bandwidth savings can be achieved by transmitting dynamic point clouds in a lower temporal resolution, and recreating a higher temporal resolution on the receiving side. Our interpolation architecture works by first downsampling the point clouds to a lower spatial resolution, then estimating scene flow using a newly designed neural network architecture, and finally upsampling the result back to the original spatial resolution. To improve the smoothness of the results, we additionally apply a novel technique called neighbour snapping. To be able to train and test our newly designed network, we created a synthetic point cloud data set of animated human bodies. Results from the evaluation of our architecture through a small-scale user study show the benefits of our method with respect to the state of the art in scene flow estimation for point clouds. Moreover, correlation between our user study and existing objective quality metrics confirm the need for new metrics to accurately predict the visual quality of point cloud contents

Visual Distortions in 360-degree Videos.

Omnidirectional (or 360°) images and videos are emergent signals being used in many areas, such as robotics and virtual/augmented reality. In particular, for virtual reality applications, they allow an immersive experience in which the user can interactively navigate through a scene with three degrees of freedom, wearing a head-mounted display. Current approaches for capturing, processing, delivering, and displaying 360° content, however, present many open technical challenges and introduce several types of distortions in the visual signal. Some of the distortions are specific to the nature of 360° images and often differ from those encountered in classical visual communication frameworks. This paper provides a first comprehensive review of the most common visual distortions that alter 360° signals going through the different processing elements of the visual communication pipeline. While their impact on viewers' visual perception and the immersive experience at large is still unknown-thus, it is an open research topic-this review serves the purpose of proposing a taxonomy of the visual distortions that can be encountered in 360° signals. Their underlying causes in the end-to-end 360° content distribution pipeline are identified. This taxonomy is essential as a basis for comparing different processing techniques, such as visual enhancement, encoding, and streaming strategies, and allowing the effective design of new algorithms and applications. It is also a useful resource for the design of psycho-visual studies aiming to characterize human perception of 360° content in interactive and immersive applications

ALTMM 2018 - 3rd international workshop on multimedia alternate realities

AltMM 2018 is the 3rd edition of the International Workshop on Multimedia Alternate Realities at ACM Multimedia. Our ambition remains to engage researchers and practitioners in discussions on how we can successfully create meaningful multimedia 'alternate realities' experiences. One of the main strengths of this workshop is that we combine different perspectives to explore how the synergy between multimedia technologies can foster and shape the creation of alternate realities and make their access an enriching and valuable experience

Watching videos together in social Virtual Reality: An experimental study on user’s QoE

In this paper, we describe a user study in which pairs of users watch a video trailer and interact with each other, using two social Virtual Reality (sVR) systems, as well as in a face-to-face condition. The sVR systems are: Facebook Spaces, based on puppet-like customized avatars, and a video-based sVR system using photo-realistic virtual user representations. We collect subjective and objective data to analyze users’ Quality of Experience (QoE) and compare their interaction in VR to that observed during the real-life scenario. Our results show that the experience delivered by the video-based sVR system is comparable with real-life settings, while the puppet-based avatars limit the perceived q

Measuring and understanding photo sharing experiences in social virtual reality

Millions of photos are shared online daily, but the richness of interaction compared with face-to-face (F2F) sharing is still missing. While this may change with social Virtual Reality (socialVR), we still lack tools to measure such immersive and interactive experiences. In this paper, we investigate photo sharing experiences in immersive environments, focusing on socialVR. Running context mapping (N=10), an expert creative session (N=6), and an online experience clustering questionnaire (N=20), we develop and statistically evaluate a questionnaire to measure photo sharing experiences. We then ran a controlled, within-subject study (N=26 pairs) to compare photo sharing under F2F, Skype, and Facebook Spaces. Using interviews, audio analysis, and our questionnaire, we found that socialVR can closely approximate F2F sharing. We contribute empirical findings on the immersiveness differences between digital communication media, and propose a socialVR questionnaire that can in the future generalize beyond photo sharing