Human-centric quality management of immersive multimedia applications

Augmented Reality (AR) and Virtual Reality (VR) multimodal systems are the latest trend within the field of multimedia. As they emulate the senses by means of omni-directional visuals, 360 degrees sound, motion tracking and touch simulation, they are able to create a strong feeling of presence and interaction with the virtual environment. These experiences can be applied for virtual training (Industry 4.0), tele-surgery (healthcare) or remote learning (education). However, given the strong time and task sensitiveness of these applications, it is of great importance to sustain the end-user quality, i.e. the Quality-of-Experience (QoE), at all times. Lack of synchronization and quality degradation need to be reduced to a minimum to avoid feelings of cybersickness or loss of immersiveness and concentration. This means that there is a need to shift the quality management from system-centered performance metrics towards a more human, QoE-centered approach. However, this requires for novel techniques in the three areas of the QoE-management loop (monitoring, modelling and control). This position paper identifies open areas of research to fully enable human-centric driven management of immersive multimedia. To this extent, four main dimensions are put forward: (1) Task and well-being driven subjective assessment; (2) Real-time QoE modelling; (3) Accurate viewport prediction; (4) Machine Learning (ML)-based quality optimization and content recreation. This paper discusses the state-of-the-art, and provides with possible solutions to tackle the open challenges

Architecture of a cloud-based fault-tolerant control platform for improving the QoS of social multimedia applications on SD-WAN

Social media application are becoming multimedia centric with live and stored video, audio, augmented reality, haptic, etc. emerging as the main categories of traffic. Their QoS requirements are more stringent than their legacy counterparts. At the carrier level, Software Defined – Wide Area Network (SD-WAN) is one of the promising technologies for transporting these multimedia traffic. A SD-WAN will typically have a mesh of centralized controllers managing the networking infrastructure. Reliable operations of these controllers are a key requirement for the successful operation of the WAN. Controller failure will prevent the forwarding switches from communicating with the controller. This will prevent the switches from forwarding any new traffic, as well as flow entries from existing traffic will also time out after a period bringing the network to a standstill. Rebooting a controller or starting a new one will introduce delays degrading the QoS. This research presents an architecture for handling controller failure via transparent migration of the controller load in a semi-meshed controller environment. The architecture includes a real time cloud-based centralized storage of the flow states north of the controllers and a virtualized connection management unit at the south. The results demonstrate that the proposed model can transparently handle controller failure without affecting the QoS

Construction of a Video Transmission Scenario in Software-Defined Networks for QoE Estimation

The services supported by data networks have become widespread, so the architectures of the new data networks are service-oriented. They are endowed with intelligence, flexibility, and programmability. The preceding is with the aim of providing acceptability by users. Thus, this paper presents the construction of a video transmission scenario over a software-defined network (SDN, Software-Defined Networking) using free software and modifying its behavior with background traffic, on which the Quality of Experience (QoE) is estimated. Subjective and objective metrics were used for the QoE estimation. For the first one, the Mean Opinion Score (MOS) was used, while the second one was studied with the Full Reference Image Quality Assessment (FR-IQA). Finally, a correlation between the two types of metrics was proposed

Survey on QoE/QoS Correlation Models for Video Streaming over Vehicular Ad-hoc Networks

Vehicular Ad-hoc Networks (VANETs) are a new emerging technology which has attracted enormous interest over the last few years. It enables vehicles to communicate with each other and with roadside infrastructures for many applications. One of the promising applications is multimedia services for traffic safety or infotainment. The video service requires a good quality to satisfy the end-user known as the Quality of Experience (QoE). Several models have been suggested in the literature to measure or predict this metric. In this paper, we present an overview of interesting researches, which propose QoE models for video streaming over VANETs. The limits and deficiencies of these models are identified, which shed light on the challenges and real problems to overcome in the future

QoE in IoT: a vision, survey and future directions

\ua9 The Author(s) 2021. The rapid evolution of the Internet of Things (IoT) is making way for the development of several IoT applications that require minimal or no human involvement in the data collection, transformation, knowledge extraction, and decision-making (actuation) process. To ensure that such IoT applications (we term them autonomic) function as expected, it is necessary to measure and evaluate their quality, which is challenging in the absence of any human involvement or feedback. Existing Quality of Experience (QoE) literature and most QoE definitions focuses on evaluating application quality from the lens of human receiving application services. However, in autonomic IoT applications, poor quality of decisions and resulting actions can degrade the application quality leading to economic and social losses. In this paper, we present a vision, survey and future directions for QoE research in IoT. We review existing QoE definitions followed by a survey of techniques and approaches in the literature used to evaluate QoE in IoT. We identify and review the role of data from the perspective of IoT architectures, which is a critical factor when evaluating the QoE of IoT applications. We conclude the paper by identifying and presenting our vision for future research in evaluating the QoE of autonomic IoT applications

Issues and Challenges Facing Low Latency in Tactile Internet

Tactile Internet is considered as the next step towards a revolutionary impact on the society, this is due to the introduction of different types of applications mainly the haptic ones that require strict Quality of Service guarantee especially in terms of latency. This would be a major challenge towards the design of new communication technologies and protocols in order to provide ultra-low latency. This article discusses the diverse technologies, communication protocols, and the necessary infrastructure to provide low latency based principally on the fifth generation (5G) of mobile network that is considered as the key enablers of the Tactile Internet. Furthermore, current research direction along with future challenges and open issues are discussed extensively

Immersive interconnected virtual and augmented reality : a 5G and IoT perspective

Despite remarkable advances, current augmented and virtual reality (AR/VR) applications are a largely individual and local experience. Interconnected AR/VR, where participants can virtually interact across vast distances, remains a distant dream. The great barrier that stands between current technology and such applications is the stringent end-to-end latency requirement, which should not exceed 20 ms in order to avoid motion sickness and other discomforts. Bringing AR/VR to the next level to enable immersive interconnected AR/VR will require significant advances towards 5G ultra-reliable low-latency communication (URLLC) and a Tactile Internet of Things (IoT). In this article, we articulate the technical challenges to enable a future AR/VR end-to-end architecture, that combines 5G URLLC and Tactile IoT technology to support this next generation of interconnected AR/VR applications. Through the use of IoT sensors and actuators, AR/VR applications will be aware of the environmental and user context, supporting human-centric adaptations of the application logic, and lifelike interactions with the virtual environment. We present potential use cases and the required technological building blocks. For each of them, we delve into the current state of the art and challenges that need to be addressed before the dream of remote AR/VR interaction can become reality

PRAM: Penalized Resource Allocation Method for Video Services

The human visual system response to picture quality degradation due to packet loss is very different from the responses of objective quality measures. While video quality due to packet loss may be impaired by at most for one Group of Pictures (GOP), its subjective quality degradation may last for several GOPs. This has a great impact on resource allocation strategies, which normally make decisions on instantaneous conditions of multiplexing buffer. This is because, when the perceptual impact of degraded video quality is much longer than its objective degradation period, any assigned resources to the degraded flow is wasted. This paper, through both simulations and analysis shows that, during resource allocation, if the quality of a video stream is significantly degraded, it is better to penalize this degraded flow from getting its full bandwidth share and instead assign the remaining share to other flows preventing them from undergoing quality degradation

Uma abordagem preditiva de DASH QoE baseada em aprendizado de máquina em multi-access edge computing

Orientador: Christian Rodolfo Esteve RothenbergDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de ComputaçãoResumo: O tráfego de serviços de vídeo multimídia está crescendo rapidamente nas redes móveis nos últimos anos. Os serviços de vídeo que usam técnicas de Dynamic Adaptive Streaming sobre HTTP (DASH) dominaram o tráfego total da Internet para transportar o tráfego de vídeo. Espera-se que as operadoras de rede móvel (Mobile Network Operators - MNOs) continuem atendendo a essa demanda crescente por tráfego de vídeo suportado por DASH, ao mesmo tempo em que fornecem uma alta qualidade de experiência (Quality of Experience - QoE) aos usuários finais. Além disso, as operadoras precisam ter um conhecimento claro acerca da qualidade de vídeo percebida pelos usuários finais e relacioná-la com o monitoramento em nível de rede, ou com informações de telemetria para identificação de problemas, análise da causa raiz e predição de padrões. Para garantir um gerenciamento de tráfego de rede com reconhecimento de QoE, um pré-requisito é que os MNOs monitorem o tráfego de rede passivamente e realizem medições efetivas de indicadores-chave de desempenho (Key Performance Indicators - KPIs) de QoE, como resoluções, eventos de paralisação, entre outros, que influenciam diretamente a percepção do usuário final. Muitas abordagens da literatura foram propostas para medir os KPIs com o objetivo de fornecer uma qualidade de serviço de vídeo aceitável. A maioria das soluções exige consciência de contexto do usuário final, o que não é viável do ponto de vista do MNO. No entanto, Deep Packet Inspection (DPI), outra solução mais amplamente usada para estimar os KPIs diretamente do tráfego de rede, não é mais uma solução conveniente para as operadoras devido à adoção de criptografia de streaming de vídeo fim-a-fim sobre TCP (HTTPs) e QUIC. Portanto, o aprendizado de máquina (Machine Learning - ML) passou a ser recentemente aceito como uma solução bem reconhecida para estimar KPIs de QoE, analisando os padrões de tráfego criptografados bem como estatísticas como qualidade de serviço (Quality of Service - QoS). Este trabalho apresenta uma abordagem mais refinada e leve, baseada em aprendizado de máquina, denominada Edge QoE Probe, para estimar QoE do usuário final para o serviço de vídeo DASH, monitorando passivamente o tráfego de rede criptografado na borda da rede. Nossa abordagem pode avaliar vários KPIs de QoE, como por exemplo resolução, taxa de bits, proporção de paralisação, entre outros, tanto em tempo real quanto por sessão. Além disso, neste trabalho investigamos o desempenho do vídeo DASH sobre o protocolo de transporte tradicional TCP (HTTPs) e QUIC. Para este propósito, avaliamos experimentalmente diferentes traces de rede celular em um ambiente emulado de alta fidelidade e comparamos o desempenho comportamental de algoritmos Adaptive Bitrate Streaming (ABS) considerando KPIs de QoE sobre TCP (HTTPs) e QUIC. Nossos resultados empíricos mostram que os algoritmos tradicionais de ABS usando QUIC como transporte precisariam alterações específicas para melhorar o desempenho em termos de QoE de vídeo baseados em DASHAbstract: Multimedia video services traffic is rapidly growing in mobile networks in recent years. Video services using Dynamic Adaptive Streaming over HTTP (DASH) techniques have dominated the total internet traffic to carry video traffic. Mobile Network Operators (MNOs) are expected to run on with this growing demand for DASH-supported video traffic while providing a high Quality of Experience (QoE) to the end-users. Besides, operators need to have a crystal notion of video quality perceived by the end-users and correlate them with network-level monitoring or telemetry information for problem identification, root cause analysis, and pattern prediction. To ensure QoE–aware network traffic management, a prerequisite for the MNOs is to monitor the network traffic passively and measure objective QoE Key Performance Indicators (KPIs) (such as resolutions and stalling events) effectively that directly influence end-user subjective feedback. Many literature approaches have been proposed to measure the KPIs aimed to deliver acceptable video service quality. Most of the solutions require end-user awareness, which is not viable from the MNOs' perspective. However, Deep Packet Inspection (DPI), another most widely used solution to estimate the KPIs directly from network traffic, is not a convenient solution anymore for the operators due to the adoption of end-to-end video streaming encryption over TCP (HTTPs) and QUIC transport protocol. Hence, in recent, Machine Learning (ML) has been accepted as a well-recognized solution for estimating QoE KPIs by analyzing the encrypted traffic patterns and statistics as Quality of Service (QoS). This work presents an ML-based lightweight and fine-grained Edge QoE Probe approach to estimate the end-user QoE for DASH video service by passively monitoring the encrypted network traffic on the edge of the network. Our approach can assess numerous QoE KPIs (such as resolution, bit-rate, quality switches, startup delay, and stall ratio) both in a real-time and per-session manner. Moreover, we investigate the DASH video service performance over the traditional TCP (HTTPs) and QUIC transport protocol in this work. For this purpose, we experimentally evaluate different cellular network traces in a high-fidelity emulated testbed and compare the behavioral performance of Adaptive Bitrate Streaming (ABS) algorithms considering QoE KPIs over TCP (HTTPs) and QUIC. Our empirical results show that QUIC suffers from traditional state-of-the-art ABS algorithms' ineffectiveness to improve video streaming performance without specific changesMestradoEngenharia de ComputaçãoMestre em Engenharia ElétricaFuncam

A survey on Quality of Experience of Virtual and Augmented Reality environments

Οι εφαρμογές εικονικής και επαυξημένης πραγματικότητας αποτελούν μια νέα και πολλά υποσχόμενη τεχνολογία, με εφαρμογές στην ιατρική, την εκπαίδευση, τα βιντεοπαιχνίδια, το ηλεκτρονικό εμπόριο και πολλές άλλες. Αυτή η τεχνολογία θέτει μια νέα πρόκληση για τους σχεδιαστές εφαρμογών, καθώς και για τους παρόχους υπηρεσιών δικτύου, επειδή είναι εντατική ως προς τους απαιτούμενους πόρους ώστε να είναι σε θέση να παρέχει συναρπαστική εμπειρία στους χρήστες της. Αυτή η πρόκληση γίνεται ακόμη πιο δύσκολη στα δίκτυα κινητής τηλεφωνίας, λόγω παραγόντων που είναι δύσκολο να μοντελοποιηθούν και να προβλεφθούν, όπως η κινητικότητα, η στρατηγική μεταβίβασης και η κατανομή πόρων. Αυτή η διπλωματική φιλοδοξεί να παράσχει μια ανασκόπηση των τεχνικών και μεθόδων εκτίμησης της ποιότητας της εμπειρίας (QoE) και των μεθόδων που έχουν αναπτυχθεί γύρω από αυτές τις εφαρμογές. Στην πρώτη ενότητα, εξετάζουμε τις στρατηγικές παροχής QoE για εφαρμογές εικονικής πραγματικότητας. Αυτή η ενότητα εξετάζει αρκετές περιπτώσεις εφαρμογών εικονικής πραγματικότητας, όπως ένα λογισμικό προσομοίωσης βαρέων μηχανημάτων, μια εκπαιδευτική εφαρμογή και άλλες ψηφιακές εφαρμογές εμβύθισης. Το εύρος και η ποικιλία των εφαρμογών και των μεθόδων εκτίμησης της ποιότητας της εμπειρίας οδηγούν σε αντικρουόμενα συμπεράσματα σχετικά με τις μεθόδους αξιολόγησης QoE. Στην επόμενη ενότητα αναφερόμαστε σε εφαρμογές επαυξημένης πραγματικότητας, και πάλι με μια αναφορά σε μια μεγάλη ποικιλία εφαρμογών, όπως ένας βοηθός επαυξημένης πραγματικότητας, βιντεοπαιχνίδια επαυξημένης πραγματικότητας και άλλες ψηφιακές εφαρμογές. Τα συμπεράσματα σε αυτήν την ενότητα είναι πιο ισχυρά και τα συναισθήματα των ανθρώπων μπορούν να σχηματίσουν πιο ουσιαστικά πορίσματα. Στην τελευταία ενότητα διερευνούμε την QoE σε εφαρμογές εικονικής και επαυξημένης πραγματικότητας για κινητές συσκευές και κινητά δίκτυα. Σε αυτό το μέρος ασχολούμαστε με πιο τεχνικές πτυχές όπως η διαχείριση της κινητικότητας, οι στρατηγικές μεταβίβασης και οι αλγόριθμοι κατανομής πόρων και ο αντίκτυπος που έχουν αυτοί στην εμπειρία των χρηστών.Virtual and augmented reality applications constitute a new and promising technology, with applications in medicine, education, gaming, e-commerce and many more. This technology poses a new challenge to application designers, as well as network service providers, because it is resource intensive in order to be able to provide the immersive experience to its users. This task becomes even more challenging in mobile networks, due to factors that are difficult to be modeled and predicted, such as mobility, handoff strategy and resource allocation. This thesis aspires to provide a review of Quality of Experience (QoE) estimation and provision techniques and methods that have been developed around these applications. In the first section, we review QoE provisioning strategies for virtual reality applications. This section examines some corner cases of augmented reality applications, such as a heavy machinery simulation software, an educational application, and many digital immersive applications. The scope and diversity of applications and implementation methods lead to some conflicting conclusions in relation to QoE evaluation methods. In the next section we refer to augmented reality applications, again with a reference to a wide variety of applications such as an augmented reality task assistant, augmented reality video games and digital immersive applications. The conclusions in this section are more robust and peoples’ feelings can form more meaningful aggregations. In the last section we investigate the QoE in virtual and augmented reality applications when these applications are implemented in mobile devices. Τhis part is concerned with more technical aspects such as mobility management, handoff strategies and resource allocation algorithms and their impact on users’ experience