Streaming and User Behaviour in Omnidirectional Videos

Omnidirectional videos (ODVs) have gone beyond the passive paradigm of traditional video, offering higher degrees of immersion and interaction. The revolutionary novelty of this technology is the possibility for users to interact with the surrounding environment, and to feel a sense of engagement and presence in a virtual space. Users are clearly the main driving force of immersive applications and consequentially the services need to be properly tailored to them. In this context, this chapter highlights the importance of the new role of users in ODV streaming applications, and thus the need for understanding their behaviour while navigating within ODVs. A comprehensive overview of the research efforts aimed at advancing ODV streaming systems is also presented. In particular, the state-of-the-art solutions under examination in this chapter are distinguished in terms of system-centric and user-centric streaming approaches: the former approach comes from a quite straightforward extension of well-established solutions for the 2D video pipeline while the latter one takes the benefit of understanding users’ behaviour and enable more personalised ODV streaming

Estimation of the QoE for video streaming services based on facial expressions and gaze direction

As the multimedia technologies evolve, the need to control their quality becomes even more important making the Quality of Experience (QoE) measurements a key priority. Machine Learning (ML) can support this task providing models to analyse the information extracted by the multimedia. It is possible to divide the ML models applications in the following categories: 1) QoE modelling: ML is used to define QoE models which provide an output (e.g., perceived QoE score) for any given input (e.g., QoE influence factor). 2) QoE monitoring in case of encrypted traffic: ML is used to analyze passive traffic monitored data to obtain insight into degradations perceived by end-users. 3) Big data analytics: ML is used for the extraction of meaningful and useful information from the collected data, which can further be converted to actionable knowledge and utilized in managing QoE. The QoE estimation quality task can be carried out by using two approaches: the objective approach and subjective one. As the two names highlight, they are referred to the pieces of information that the model analyses. The objective approach analyses the objective features extracted by the network connection and by the used media. As objective parameters, the state-of-the-art shows different approaches that use also the features extracted by human behaviour. The subjective approach instead, comes as a result of the rating approach, where the participants were asked to rate the perceived quality using different scales. This approach had the problem of being a time-consuming approach and for this reason not all the users agree to compile the questionnaire. Thus the direct evolution of this approach is the ML model adoption. A model can substitute the questionnaire and evaluate the QoE, depending on the data that analyses. By modelling the human response to the perceived quality on multimedia, QoE researchers found that the parameters extracted from the users could be different, like Electroencephalogram (EEG), Electrocardiogram (ECG), waves of the brain. The main problem with these techniques is the hardware. In fact, the user must wear electrodes in case of ECG and EEG, and also if the obtained results from these methods are relevant, their usage in a real context could be not feasible. For this reason, my studies have been focused on the developing of a Machine Learning framework completely unobtrusively based on the Facial reactions

An Overview of the Networking Issues of Cloud Gaming: A Literature Review

With the increasing prevalence of video games comes innovations that aim to evolve them. Cloud gaming is poised as the next phase of gaming. It enables users to play video games on any internet-enabled device. Such improvement could, therefore, enhance the processing power of existing devices and solve the need to spend large amounts of money on the latest gaming equipment. However, others argue that it may be far from being practically functional. Since cloud gaming places dependency on networks, new issues emerge. In relation, this paper is a review of the networking perspective of cloud gaming. Specifically, the paper analyzes its issues and challenges along with possible solutions. In order to accomplish the study, a literature review was performed. Results show that there are numerous issues and challenges regarding cloud gaming networks. Generally, cloud gaming has problems with its network quality of service (QoS) and quality of experience (QoE). The poor QoS and QoE of cloud gaming can be linked to unsatisfactory latency, bandwidth, delay, packet loss, and graphics quality. Moreover, the cost of providing the service and the complexity of implementing cloud gaming were considered challenges. For these issues and challenges, solutions were found. The solutions include lag or latency compensation, compression with encoding techniques, client computing power, edge computing, machine learning, frame adaption, and GPU-based server selection. However, these have limitations and may not always be applicable. Thus, even if solutions exist, it would be beneficial to analyze the networking side of cloud gaming further

Collaborative interaction in immersive 360º experiences

Os sistemas de reprodução de vídeo tornaram-se, a cada dia, mais habituais e utilizados. Consequentemente, foram criadas extensões desta tecnologia permitindo colaboração multipessoal de modo a poder assistir remotamente e sincronamente. Exemplos conhecidos são o Watch2gether, Sync Video e Netflix Party, que nos permitem assistir vídeos síncrona e remotamente com amigos. Estas aplicações de visualização conjunta, apesar de bem desenvolvidas, estão limitadas ao típico formato, não se estendendo a vídeos 360. O principal objetivo deste projeto é então expandir a pesquisa nesta área ao desenvolver um sistema colaborativo para vídeos 360. Já foram direcionados vários esforços na área de vídeos 360o, um deles sendo o projeto AV360, aplicação que permite ao utilizador visualizar e editar este tipo de vídeos com anotações e guias. O sistema que se pretende integrar é um seguimento ao AV360, utilizando parte das tecnologias já implementadas. De maneira a compartimentalizar e facilitar a pesquisa são considerados os seguintes temas de forma individual: a visualização de vídeos 360o, a generalidade dos sistemas colaborativos, a aplicação de colaboração em ambientes virtuais e os sistemas de vídeo colaborativos. É importante ter noção das vantagens e desvantagens de assistir a um vídeo 360o, conseguir retirar o que é a essência nestes vídeos e mantê-la, integrando também a inclusão de outros utilizadores. Na escolha das atividades colaborativas a aplicar, é imprescindível analisar o estado em que os sistemas colaborativos se encontram hoje em dia e posteriormente afunilar a pesquisa para a colaboração em ambientes virtuais e em vídeos. Dentro de todos os métodos analisados só os adaptáveis a ambientes imersivos e a vídeos são escolhidos e desenvolvidos neste projeto. Com base numa pesquisa aprofundada sobre o assunto, é criado um sistema de colaboração em vídeos 360o. O software permite que os utilizadores assistam em simultâneo a um vídeo enquanto comunicam de uma forma verbal e não verbal para se expressarem e partilharem a experiência do momento. Este trabalho tem em mente que parte das ideias implementadas possam ser reutilizáveis para outros projetos de experiências imersivas