Quality of experience evaluation of voice communication: an affect-based approach

Voice communication systems such as Voice-over IP (VoIP), Public Switched Telephone Networks, and Mobile Telephone Networks, are an integral means of human tele-interaction. These systems pose distinctive challenges due to their unique characteristics such as low volume, burstiness and stringent delay/loss requirements across heterogeneous underlying network technologies. Effective quality evaluation methodologies are important for system development and refinement, particularly by adopting user feedback based measurement. Presently, most of the evaluation models are system-centric (Quality of Service or QoS-based), which questioned us to explore a user-centric (Quality of Experience or QoE-based) approach as a step towards the human-centric paradigm of system design. We research an affect-based QoE evaluation framework which attempts to capture users\u27 perception while they are engaged in voice communication. Our modular approach consists of feature extraction from multiple information sources including various affective cues and different classification procedures such as Support Vector Machines (SVM) and k-Nearest Neighbor (kNN). The experimental study is illustrated in depth with detailed analysis of results. The evidences collected provide the potential feasibility of our approach for QoE evaluation and suggest the consideration of human affective attributes in modeling user experience

"I'm the Jedi!" - A Case Study of User Experience in 3D Tele-immersive Gaming

Abstract—In this paper, we present the results from a quantitative and qualitative study of distributed gaming in 3D tele-immersive (3DTI) environments. We explore the Qual-ity of Experience (QoE) of users in the new cyber-physical gaming environment. Guided by a theoretical QoE model, we conducted a case study and evaluated the impact of various Quality of Service (QoS) metrics (e.g., end-to-end delay, visual quality, etc.) on 3DTI gaming experience. We also identified a number of non-technical factors that are not captured by the original theoretical model, such as age, social interaction, and physical setup. Our analysis highlights new implications for the next-generation gaming system design, as well as a more comprehensive conceptual framework that captures non-technical influences for user experience in such environments

Human-centric control of video functions and underlying resources in 3D tele-immersive systems

3D tele-immersion (3DTI) has the potential of enabling virtual-reality-like interaction among remote people with real-time 3D video. However, today's 3DTI systems still suffer from various performance issues, limiting their broader deployment, due to the enormous demand on temporal (computing) and spatial (networking) resources. Past research focused on system-centric approaches for technical optimization, without taking human users into the loop. We argue that human factors (including user preferences, semantics, limitations, etc.) are an important and integral part of the cyber-physical 3DTI systems, and should not be neglected. This thesis proposes a novel, comprehensive, human-centric framework for improving the qualities of 3DTI throughout its video function pipeline. We make three major contributions at different phases of the pipeline. At the sending side, we develop an intra-stream data adaptation scheme that reduces level-of-details within each stream without users being aware of it. This human-centric approach exploits limitations of human vision, and excludes details that are imperceptible. It effectively alleviates the data load for computation-intensive operations, thus improves the temporal efficiency of the systems. Yet even with intra-stream data reduced, spatial efficiency is still a problem due to the multi-stream/multi-site nature of 3DTI collaboration. We thus develop an inter-stream data adaptation scheme at the networking phase to reduce the number of streams with minimal disruption to the visual quality. This human-centric approach prioritizes streams based on user views and excludes less important streams from transmission. It considerably reduces the data load for networking, and thus enhances the spatial resource efficiency. The above two approaches (level-of-details reduction within a video stream and view-based differentiation among streams) work seamlessly together to bring both temporal and spatial resource demands under control, and prove to improve various qualities of the systems. Finally, at the receiving side, we take a holistic approach to study the ``quality'' concept in 3DTI environments. Our human-centric quality framework focuses on the Quality-of-Experience (QoE) concept that models user's perceptions, emotions, performances, etc. It investigates how the traditional Quality-of-Service (QoS) impacts QoE, and reveals how QoS should be improved for the best user experience. This thesis essentially demonstrates the importance of bringing human-awareness into the design, execution, and evaluation of the complex resource-constrained 3DTI environments

Towards Multi-Site Collaboration in 3D Tele-Immersive Environments

3D tele-immersion (3DTI) has recently emerged as a new way of video- mediated collaboration across the Internet. Unlike conventional 2D video-conferencing systems, it can immerse remote users into a shared 3D virtual space so that they can interact or collaborate "virtually". However, most existing 3DTI systems can support only two sites of collaboration, due to the huge demand of networking resources and the lack of a simple yet efficient data dissemination model. In this paper, we propose a general publish-subscribe model for multi-site 3DTI systems, which efficiently utilizes limited network resources by leveraging user interest. We focus on the overlay construction problem in the publish-subscribe model by exploring a spectrum of heuristic algorithms for data dissemination. With extensive simulation, we identify the advantages of a simple randomized algorithm. We propose optimization to further improve the randomized algorithm by exploiting semantic correlation. Experimental results demonstrate that we can achieve an improvement by a factor of five

Understanding the Human Perceptions in Tele-Immersive Shared Activity

Both comparative category rating (CCR) and degradation category rating (DCR) methods have been heavily employed in the subjective evaluations of media systems. The resulting metrics, comparative mean-opinion-score (CMOS) and degradation mean-opinion-score (DMOS), can be used to describe the system subjective quality. However, the subjective metrics may work unsuccessfully when the variance of participant votes is large. The diversity in human interests can appear due to the tradeoffs of multiple quality dimensions, which concurrently dominate the overall quality of the media system. In this paper, we conduct a user study with 19 participants to evaluate the subjective quality of two tele-immersive shared activities (TISA), where media samples of different qualities are evaluated in case of each activity. Our study aims at (1) showing the effectiveness and limitation of CMOS and DMOS using real subjective data, and (2) demonstrating the heterogeneous impacts of TISAs on human perceptions.NSF CNS 0834480, 0964081, 1012194, IIP 1110178published or submitted for publicationnot peer reviewe

Dynamic overlay multicast in 3D video collaborative systems

Multi-stream/multi-site 3D video collaborative systems are promising as they enable remote users to interact in a 3D vir-tual space with a sense of co-presence. However, the decen-tralized content dissemination remains a challenge. In this work, we explore approaches to construct adaptive overlay based on the users ’ visual interest in the collaborative space. Particularly, we consider the practical challenge that the user’s interest might change dynamically. We propose, com-pare, and evaluate three algorithms to handle the view dy-namics. With extensive experiments, we demonstrate that an algorithm that exploits view locality can achieve efficient bandwidth utilization, high topology stability, and great scalability

Collaborative Dancing in Tele-immersive Environment

We first present the tele-immersive environments developed jointly by University of Illinois at Urbana-Champaign and University of California at Berkeley. The environment features 3D full and real body capturing, wide field of view, multi-display 3D rendering, and attachment free participant. We then describe a study of collaborative dancing between remotely located dancers in the shared virtual space. Two professional dancers are invited to the tele-immersive site of each university. As a preliminary experiment, we let the dancers perform elementary body movements and coordinate their dancing. The coordination requires one dancer to take the lead while the other follows her by appropriate movements. During the experiment, the dancers are dancing at various motion rates to evaluate how well the collaborative dancing is supported with the current technical boundary. Our important findings indicate that 1) tele-immersive environments have strong potential impact on the concept of choreography and communication of live dance performance, 2) the presence of multi-display system, real body 3D rendering, audio channel, and less intrusiveness greatly enhances the immersive and dancing experience, and 3) the level of synchronization achieved by the dancers is higher than that expected from the video rate