From QoS Distributions to QoE Distributions: a System's Perspective

In the context of QoE management, network and service providers commonly rely on models that map system QoS conditions (e.g., system response time, paket loss, etc.) to estimated end user QoE values. Observable QoS conditions in the system may be assumed to follow a certain distribution, meaning that different end users will experience different conditions. On the other hand, drawing from the results of subjective user studies, we know that user diversity leads to distributions of user scores for any given test conditions (in this case referring to the QoS parameters of interest). Our previous studies have shown that to correctly derive various QoE metrics (e.g., Mean Opinion Score (MOS), quantiles, probability of users rating "good or better", etc.) in a system under given conditions, there is a need to consider rating distributions obtained from user studies, which are often times not available. In this paper we extend these findings to show how to approximate user rating distributions given a QoS-to-MOS mapping function and second order statistics. Such a user rating distribution may then be combined with a QoS distribution observed in a system to finally derive corresponding distributions of QoE scores. We provide two examples to illustrate this process: 1) analytical results using a Web QoE model relating waiting times to QoE, and 2) numerical results using measurements relating packet losses to video stall pattern, which are in turn mapped to QoE estimates. The results in this paper provide a solution to the problem of understanding the QoE distribution in a system, in cases where the necessary data is not directly available in the form of models going beyond the MOS, or where the full details of subjective experiments are not available.Comment: 4th International Workshop on Quality of Experience Management (QoE Management 2020), featured by IEEE Conference on Network Softwarization (IEEE NetSoft 2020), Ghent, Belgiu

Fundamental Relationships for Deriving QoE in Systems

In the context of subjective user studies conducted to derive relationships between influence factors and QoE, user diversity leads to distributions of user scores for test conditions. Such models are commonly exploited by service/network providers to derive various QoE metrics in their system, such as expected QoE, or the percentage of users rating above a certain threshold. The question arises as to how to combine a) user rating distributions obtained from subjective studies, and b) system performance condition distributions, so as to obtain the actual observed QoE distribution in the system? Moreover, how can various QoE metrics of interest in the system be derived? We prove a fundamental relationship showing that the expected system QoE is equal to the expected Mean Opinion Score (MOS) in the system. While subjective user studies commonly report only QoS-to-MOS mapping functions, we show that to derive additional QoE metrics in the system, it is necessary to use corresponding QoS-to-QoE metric mapping functions (beyond only QoS-to-MOS) as derived from user rating distributions in subjective studies. The results of the paper provide important insights for deriving QoE metrics from a systems perspective