Quality of experience in telemeetings and videoconferencing: a comprehensive survey

Telemeetings such as audiovisual conferences or virtual meetings play an increasingly important role in our professional and private lives. For that reason, system developers and service providers will strive for an optimal experience for the user, while at the same time optimizing technical and financial resources. This leads to the discipline of Quality of Experience (QoE), an active field originating from the telecommunication and multimedia engineering domains, that strives for understanding, measuring, and designing the quality experience with multimedia technology. This paper provides the reader with an entry point to the large and still growing field of QoE of telemeetings, by taking a holistic perspective, considering both technical and non-technical aspects, and by focusing on current and near-future services. Addressing both researchers and practitioners, the paper first provides a comprehensive survey of factors and processes that contribute to the QoE of telemeetings, followed by an overview of relevant state-of-the-art methods for QoE assessment. To embed this knowledge into recent technology developments, the paper continues with an overview of current trends, focusing on the field of eXtended Reality (XR) applications for communication purposes. Given the complexity of telemeeting QoE and the current trends, new challenges for a QoE assessment of telemeetings are identified. To overcome these challenges, the paper presents a novel Profile Template for characterizing telemeetings from the holistic perspective endorsed in this paper

ViCoCoS-3D: Videoconferencing common scenes

This paper presents a 3D video dataset containing sequences with typical content from videoconferencing scenarios. The objective of this dataset is to provide freely-available sequences for the research community to support the develop-ment and evaluation of processing techniques applicable to 3D videoconferencing systems. Therefore, a detailed description of the generation process and the content characteristics is provided, together with insights of possible applications of the datase

A QoE study of different stream and layout configurations in video conferencing under limited network conditions

One particular problem of QoE research in video conferencing is, that most research in the past concentrated on one-to-one video conferencing or simply video consumption. However, video conferencing with two people (one-to-one) and within a group (multi-party) is different. Particularly, limitations of one participant might have an effect on the QoE of the whole group. This possible effect however is not well studied. Therefore, this paper aims to better understand the impact of individual limitations towards the groups QoE. To do so, we show a study about different video stream configurations and layouts for multi-party conferencing in respect to individual network limitations. For this, we conduct a user study with 20 participants in 5 groups, in a semi-controlled setup. Such a setup, combines supervising participants locally while still using our software infrastructure deployed in the internet. Furthermore, we use an asymmetric experiment design, by putting every participant under a different condition, as this proposes a more realistic scenario. Within our study, we look at three different factors: layout, video quality and network limitations. To foster conversation between participants, the group engaged in a discussion about different survival questions. Our findings show that packet loss and the resulting distortions have a greater impact on the QoE as reducing the video quality by its resolution. Furthermore, our findings indicate that participants are more satisfied in a visually equal layout (showing participants in a similar size) and a more balanced stream configuration

1Mbps is enough: video quality and individual idiosyncrasies in multiparty HD video-conferencing

Most video platforms deliver HD video in high bitrate encoding. Modern video-conferencing systems are capable of handling HD streams, but using multiparty conferencing, average internet connections in the home are on their bandwidth limit. For properly managing the encoding bitrate in videoconferencing, we must know what is the minimum bitrate requirement to provide users an acceptable experience, and what is the bitrate level after which QoE saturates?. Most available subjective studies in this area used rather dated technologies. We report on a multiparty study on video quality with HD resolution. We tested different encoding bitrates (256kbs, 1024kbs and 4096kbs) and packet loss rates (0, 0.5%) in groups of 4 participants with a

A requirement analysis for a multi-party conferencing testbed

Current videoconferencing services such as Skype and Google+Hangouts provide mechanisms for engaging in multi-party conversations. Although these services provide basic support, they lack functionalities that take into account the users roles and context. Currently, the multimedia research community is actively engaged in conducting experiments concerning Quality of Experi- ence (QoE). This paper provides a requirement analy- sis for a multi-party conferencing testbed, that is de- signed for conducting controlled telecommunication ex- periments for assessing QoE. A pre-study, in the form of an online survey, investigated the experience with pre- vious tools and identified the interest towards using the CWI tool for future studies. Requirements are derived through semi-structured interviews by looking into the experimental process and issues that stakeholders are currently facing. Results show that having the capabil- ity to pre-define the experimental conditions and man- ually adjust these throughout the experiment are inte- gral aspects within the tool. Furthermore, various con- trol possibilities to interact with the test participants are needed. Subjective assessment integration in the form of questionnaires and logging of technical condi- tions are important requirements to support the anal- ysis phase. Documentation, coding support and easy customizability are crucial aspects influencing the over- all tool usability. The listed requirements provide a framework for further development of QoE assessment tools in the area of telecommunication studies and, furthermore, contribute to the open-source development of the multi-party conferencing testbed

QoE Estimation of WebRTC-based Audio-visual Conversations from Facial and Speech Features

The utilization of user’s facial- and speech-related features for the estimation of the Quality of Experience (QoE) of multimedia services is still underinvestigated despite its potential. Currently, only the use of either facial or speech features individually has been proposed, and relevant limited experiments have been performed. To advance in this respect, in this study, we focused on WebRTC-based videoconferencing, where it is often possible to capture both the facial expressions and vocal speech characteristics of the users. First, we performed thorough statistical analysis to identify the most significant facial- and speech-related features for QoE estimation, which we extracted from the participants’ audio-video data collected during a subjective assessment. Second, we trained individual QoE estimation machine learning-based models on the separated facial and speech datasets. Finally, we employed data fusion techniques to combine the facial and speech datasets into a single dataset to enhance the QoE estimation performance due to the integrated knowledge provided by the fusion of facial and speech features. The obtained results demonstrate that the data fusion technique based on the Improved Centered Kernel Alignment (ICKA) allows for reaching a mean QoE estimation accuracy of 0.93, whereas the values of 0.78 and 0.86 are reached when using only facial or speech features, respectively

The contrast effect: QoE of mixed video-qualities at the same time

In desktop multi-party video-conferencing videostreams of participants are delivered in different qualities, but we know little about how such composition of the screen affects the quality of experience. Do the different videostreams serve as indirect quality references and the perceived video quality is thus dependent on other streams in the same session? How is the relation between the perceived qualities of each stream and the perceived quality of the overall session? To answer these questions we conducted a crowdsourcing study, in which we gathered over 5000 perceived quality ratings of overall sessions and individual streams. Our results show a contrast effect: high quality streams are rated better when more low quality streams are co-present, and vice versa. In turn, the quality p

ReCoCo: Reinforcement learning-based Congestion control for Real-time applications

Real-time communication (RTC) platforms have seen a considerable surge in popularity in recent years, largely due to the COVID-19 pandemic which facilitated remote work. To ensure adequate Quality of Experience (QoE) for users, a good congestion control algorithm is needed. RTC applications use UDP, so congestion control is done on the application layer, leaving way for advanced algorithms. In this paper, we propose ReCoCo, a solution for congestion control in RTC applications based on Reinforcement learning (RL). ReCoCo gains information about the network conditions at the receiver-side, such as receiving rate, one-way delay and loss ratio and predicts the available bandwidth in the next time bin. We train ReCoCo on 9 bandwidth trace files that cover a vast array of network types. We try different algorithms, states and parameters, training both specific and general models. We find that ReCoCo outperforms the de-facto standard heuristic algorithm GCC in both specialized and general models. We also make observations on the difficulty of generalization when using RL