Discrete-time Analysis of Multicomponent GI/GI/1 Queueing Networks

In this work, we provide initial insights regarding the error introducedinto multicomponent queueing systems by assuming the departure processes of arbitraryGI/GI/1-oo queues to be renewal processes. To this end, we compute the sojourntime distribution as well as departure distributions of a linear chain of queueingcomponents and compare the results to a simulation of the same system. By applyingthe renewal approximation, potential autocorrelations of the departure processesare lost. We investigate the magnitude of this error regarding both the sojourn timeas well as interdeparture time distributions for a broad set of parameters. Althoughmore indepth studies are needed, our results show that both distributions can beclosely approximated, which allows the application of the model to asses the performanceof real world NFV function chains

Power Reduction Opportunities on End-User Devices in Quality-Steady Video Streaming

This paper uses a crowdsourced dataset of online video streaming sessions to investigate opportunities to reduce the power consumption while considering QoE. For this, we base our work on prior studies which model both the end-user's QoE and the end-user device's power consumption with the help of high-level video features such as the bitrate, the frame rate, and the resolution. On top of existing research, which focused on reducing the power consumption at the same QoE optimizing video parameters, we investigate potential power savings by other means such as using a different playback device, a different codec, or a predefined maximum quality level. We find that based on the power consumption of the streaming sessions from the crowdsourcing dataset, devices could save more than 55% of power if all participants adhere to low-power settings.Comment: 4 pages, 3 figure

How do crowdworker communities and microtask markets influenceeach other? a data-driven study on amazon mechanical turk

Crowdworker online communities -- operating in fora like mTurkForum and TurkerNation -- are an important actor in microwork markets. Albeit central to market dynamics, how the behavior of crowdworker communities and the dynamics of online marketplaces influence each other is yet to be understood. To provide quantitative evidence of such influence, we performed an analysis on 6-years worth of mTurk market activities and community discussions in six fora. We investigated the nature of the relationships that exist between activities in fora, tasks published in mTurk, requesters for such tasks, and task completion speed. We validate -- and expand upon -- results from previous work by showing that (i) there are differences between market demand and community activities that are specific to fora and task types; (ii) the temporal progression of HIT availability in the market is predictive of the upcoming amount of crowdworker discussions, with significant differences across fora and discussion categories; (iii) activities in fora can have a significant positive impact on the completion speed of tasks available in the market

From QoS Distributions to QoE Distributions: a System's Perspective

In the context of QoE management, network and service providers commonly rely on models that map system QoS conditions (e.g., system response time, paket loss, etc.) to estimated end user QoE values. Observable QoS conditions in the system may be assumed to follow a certain distribution, meaning that different end users will experience different conditions. On the other hand, drawing from the results of subjective user studies, we know that user diversity leads to distributions of user scores for any given test conditions (in this case referring to the QoS parameters of interest). Our previous studies have shown that to correctly derive various QoE metrics (e.g., Mean Opinion Score (MOS), quantiles, probability of users rating "good or better", etc.) in a system under given conditions, there is a need to consider rating distributions obtained from user studies, which are often times not available. In this paper we extend these findings to show how to approximate user rating distributions given a QoS-to-MOS mapping function and second order statistics. Such a user rating distribution may then be combined with a QoS distribution observed in a system to finally derive corresponding distributions of QoE scores. We provide two examples to illustrate this process: 1) analytical results using a Web QoE model relating waiting times to QoE, and 2) numerical results using measurements relating packet losses to video stall pattern, which are in turn mapped to QoE estimates. The results in this paper provide a solution to the problem of understanding the QoE distribution in a system, in cases where the necessary data is not directly available in the form of models going beyond the MOS, or where the full details of subjective experiments are not available.Comment: 4th International Workshop on Quality of Experience Management (QoE Management 2020), featured by IEEE Conference on Network Softwarization (IEEE NetSoft 2020), Ghent, Belgiu

Impact of the Number of Votes on the Reliability and Validity of Subjective Speech Quality Assessment in the Crowdsourcing Approach

The subjective quality of transmitted speech is traditionally assessed in a controlled laboratory environment according to ITU-T Rec. P.800. In turn, with crowdsourcing, crowdworkers participate in a subjective online experiment using their own listening device, and in their own working environment. Despite such less controllable conditions, the increased use of crowdsourcing micro-task platforms for quality assessment tasks has pushed a high demand for standardized methods, resulting in ITU-T Rec. P.808. This work investigates the impact of the number of judgments on the reliability and the validity of quality ratings collected through crowdsourcing-based speech quality assessments, as an input to ITU-T Rec. P.808 . Three crowdsourcing experiments on different platforms were conducted to evaluate the overall quality of three different speech datasets, using the Absolute Category Rating procedure. For each dataset, the Mean Opinion Scores (MOS) are calculated using differing numbers of crowdsourcing judgements. Then the results are compared to MOS values collected in a standard laboratory experiment, to assess the validity of crowdsourcing approach as a function of number of votes. In addition, the reliability of the average scores is analyzed by checking inter-rater reliability, gain in certainty, and the confidence of the MOS. The results provide a suggestion on the required number of votes per condition, and allow to model its impact on validity and reliability.Comment: This paper has been accepted for publication in the 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX

Survey of Web-based Crowdsourcing Frameworks for Subjective Quality Assessment

The popularity of the crowdsourcing for performing various tasks online increased significantly in the past few years. The low cost and flexibility of crowdsourcing, in particular, attracted researchers in the field of subjective multimedia evaluations and Quality of Experience (QoE). Since online assessment of multimedia content is challenging, several dedicated frameworks were created to aid in the designing of the tests, including the support of the testing methodologies like ACR, DCR, and PC, setting up the tasks, training sessions, screening of the subjects, and storage of the resulted data. In this paper, we focus on the web-based frameworks for multimedia quality assessments that support commonly used crowdsourcing platforms such as Amazon Mechanical Turk and Microworkers. We provide a detailed overview of the crowdsourcing frameworks and evaluate them to aid researchers in the field of QoE assessment in the selection of frameworks and crowdsourcing platforms that are adequate for their experiments

TOWARDS A COMPREHENSIVE FRAMEWORK FOR QOE AND USER BEHAVIOR MODELLING

ABSTRACT While the modeling of QoE has made significant advances over the last couple of years, currently existing models still lack an integration of user behavior aspects and user context factors along with the consideration of appropriate temporal scales. Therefore, the goal of this paper is to present a comprehensive QoE and user behavior model providing a framework which allows joining a multitude of existing modeling approaches under the perspectives of service provider benefit, user well-being and technical system performance. In addition, we discuss the role of a broad range of corresponding influence factors, with a specific emphasis on user and context issues, and illustrate our proposal through a series of related use cases

Pervasive Communities in the Internet of People

Peer reviewe