Quality of experience evaluation of voice communication: an affect-based approach

Voice communication systems such as Voice-over IP (VoIP), Public Switched Telephone Networks, and Mobile Telephone Networks, are an integral means of human tele-interaction. These systems pose distinctive challenges due to their unique characteristics such as low volume, burstiness and stringent delay/loss requirements across heterogeneous underlying network technologies. Effective quality evaluation methodologies are important for system development and refinement, particularly by adopting user feedback based measurement. Presently, most of the evaluation models are system-centric (Quality of Service or QoS-based), which questioned us to explore a user-centric (Quality of Experience or QoE-based) approach as a step towards the human-centric paradigm of system design. We research an affect-based QoE evaluation framework which attempts to capture users\u27 perception while they are engaged in voice communication. Our modular approach consists of feature extraction from multiple information sources including various affective cues and different classification procedures such as Support Vector Machines (SVM) and k-Nearest Neighbor (kNN). The experimental study is illustrated in depth with detailed analysis of results. The evidences collected provide the potential feasibility of our approach for QoE evaluation and suggest the consideration of human affective attributes in modeling user experience

Survey on QoE/QoS Correlation Models for Video Streaming over Vehicular Ad-hoc Networks

Vehicular Ad-hoc Networks (VANETs) are a new emerging technology which has attracted enormous interest over the last few years. It enables vehicles to communicate with each other and with roadside infrastructures for many applications. One of the promising applications is multimedia services for traffic safety or infotainment. The video service requires a good quality to satisfy the end-user known as the Quality of Experience (QoE). Several models have been suggested in the literature to measure or predict this metric. In this paper, we present an overview of interesting researches, which propose QoE models for video streaming over VANETs. The limits and deficiencies of these models are identified, which shed light on the challenges and real problems to overcome in the future

Context aware advertising

IP Television (IPTV) has created a new arena for digital advertising that has not been explored to its full potential yet. IPTV allows users to retrieve on demand content and recommended content; however, very limited research has been applied in the domain of advertising in IPTV systems. The diversity of the field led to a lot of mature efforts in the fields of content recommendation and mobile advertising. The introduction of IPTV and smart devices led to the ability to gather more context information that was not subject of study before. This research attempts at studying the different contextual parameters, how to enrich the advertising context to tailor better ads for users, devising a recommendation engine that utilizes the new context, building a prototype to prove the viability of the system and evaluating it on different quality of service and quality of experience measures. To tackle this problem, a review of the state of the art in the field of context-aware advertising as well as the related field of context-aware multimedia have been studied. The intent was to come up with the most relevant contextual parameters that can possibly yield a higher percentage precision for recommending advertisements to users. Subsequently, a prototype application was also developed to validate the feasibility and viability of the approach. The prototype gathers contextual information related to the number of viewers, their age, genders, viewing angles as well as their emotions. The gathered context is then dispatched to a web service which generates advertisement recommendations and sends them back to the user. A scheduler was also implemented to identify the most suitable time to push advertisements to users based on their attention span. To achieve our contributions, a corpus of 421 ads was gathered and processed for streaming. The advertisements were displayed in reality during the holy month of Ramadan, 2016. A data gathering application was developed where sample users were presented with 10 random ads and asked to rate and evaluate the advertisements according to a predetermined criteria. The gathered data was used for training the recommendation engine and computing the latent context-item preferences. This also served to identify the performance of a system that randomly sends advertisements to users. The resulting performance is used as a benchmark to compare our results against. When it comes to the recommendation engine itself, several implementation options were considered that pertain to the methodology to create a vector representation of an advertisement as well as the metric to use to measure the similarity between two advertisement vectors. The goal is to find a representation of advertisements that circumvents the cold start problem and the best similarity measure to use with the different vectorization techniques. A set of experiments have been designed and executed to identify the right vectorization methodology and similarity measure to apply in this problem domain. To evaluate the overall performance of the system, several experiments were designed and executed that cover different quality aspects of the system such as quality of service, quality of experience and quality of context. All three aspects have been measured and our results show that our recommendation engine exhibits a significant improvement over other mechanisms of pushing ads to users that are employed in currently existing systems. The other mechanisms placed in comparison are the random ad generation and targeted ad generation. Targeted ads mechanism relies on demographic information of the viewer with disregard to his/her historical consumption. Our system showed a precision percentage of 69.70% which means that roughly 7 out of 10 recommended ads are actually liked and viewed to the end by the viewer. The practice of randomly generating ads yields a result of 41.11% precision which means that only 4 out of 10 recommended ads are actually liked by viewers. The targeted ads system resulted in 51.39% precision. Our results show that a significant improvement can be introduced when employing context within a recommendation engine. When introducing emotion context, our results show a significant improvement in case the user’s emotion is happiness; however, it showed a degradation of performance when the user’s emotion is sadness. When considering all emotions, the overall results did not show a significant improvement. It is worth noting though that ads recommended based on detected emotions using our systems proved to always be relevant to the user\u27s current mood

Emotion Communication System

In today's increasingly rich material life, people are shifting their focus from the physical world to the spiritual world. In order to identify and care for people's emotions, human-machine interaction systems have been created. The currently available human-machine interaction systems often support the interaction between human and robot under the line-of-sight (LOS) propagation environment, while most communications in terms of human-to-human and human-to-machine are non-LOS (NLOS). In order to break the limitation of the traditional human–machine interaction system, we propose the emotion communication system based on NLOS mode. Specifically, we first define the emotion as a kind of multimedia which is similar to voice and video. The information of emotion can not only be recognized, but can also be transmitted over a long distance. Then, considering the real-time requirement of the communications between the involved parties, we propose an emotion communication protocol, which provides a reliable support for the realization of emotion communications. We design a pillow robot speech emotion communication system, where the pillow robot acts as a medium for user emotion mapping. Finally, we analyze the real-time performance of the whole communication process in the scene of a long distance communication between a mother-child users' pair, to evaluate the feasibility and effectiveness of emotion communications

Browse-to-search

This demonstration presents a novel interactive online shopping application based on visual search technologies. When users want to buy something on a shopping site, they usually have the requirement of looking for related information from other web sites. Therefore users need to switch between the web page being browsed and other websites that provide search results. The proposed application enables users to naturally search products of interest when they browse a web page, and make their even causal purchase intent easily satisfied. The interactive shopping experience is characterized by: 1) in session - it allows users to specify the purchase intent in the browsing session, instead of leaving the current page and navigating to other websites; 2) in context - -the browsed web page provides implicit context information which helps infer user purchase preferences; 3) in focus - users easily specify their search interest using gesture on touch devices and do not need to formulate queries in search box; 4) natural-gesture inputs and visual-based search provides users a natural shopping experience. The system is evaluated against a data set consisting of several millions commercial product images. © 2012 Authors

Quality of experience in telemeetings and videoconferencing: a comprehensive survey

Telemeetings such as audiovisual conferences or virtual meetings play an increasingly important role in our professional and private lives. For that reason, system developers and service providers will strive for an optimal experience for the user, while at the same time optimizing technical and financial resources. This leads to the discipline of Quality of Experience (QoE), an active field originating from the telecommunication and multimedia engineering domains, that strives for understanding, measuring, and designing the quality experience with multimedia technology. This paper provides the reader with an entry point to the large and still growing field of QoE of telemeetings, by taking a holistic perspective, considering both technical and non-technical aspects, and by focusing on current and near-future services. Addressing both researchers and practitioners, the paper first provides a comprehensive survey of factors and processes that contribute to the QoE of telemeetings, followed by an overview of relevant state-of-the-art methods for QoE assessment. To embed this knowledge into recent technology developments, the paper continues with an overview of current trends, focusing on the field of eXtended Reality (XR) applications for communication purposes. Given the complexity of telemeeting QoE and the current trends, new challenges for a QoE assessment of telemeetings are identified. To overcome these challenges, the paper presents a novel Profile Template for characterizing telemeetings from the holistic perspective endorsed in this paper

A Human-Centric Metaverse Enabled by Brain-Computer Interface: A Survey

The growing interest in the Metaverse has generated momentum for members of academia and industry to innovate toward realizing the Metaverse world. The Metaverse is a unique, continuous, and shared virtual world where humans embody a digital form within an online platform. Through a digital avatar, Metaverse users should have a perceptual presence within the environment and can interact and control the virtual world around them. Thus, a human-centric design is a crucial element of the Metaverse. The human users are not only the central entity but also the source of multi-sensory data that can be used to enrich the Metaverse ecosystem. In this survey, we study the potential applications of Brain-Computer Interface (BCI) technologies that can enhance the experience of Metaverse users. By directly communicating with the human brain, the most complex organ in the human body, BCI technologies hold the potential for the most intuitive human-machine system operating at the speed of thought. BCI technologies can enable various innovative applications for the Metaverse through this neural pathway, such as user cognitive state monitoring, digital avatar control, virtual interactions, and imagined speech communications. This survey first outlines the fundamental background of the Metaverse and BCI technologies. We then discuss the current challenges of the Metaverse that can potentially be addressed by BCI, such as motion sickness when users experience virtual environments or the negative emotional states of users in immersive virtual applications. After that, we propose and discuss a new research direction called Human Digital Twin, in which digital twins can create an intelligent and interactable avatar from the user's brain signals. We also present the challenges and potential solutions in synchronizing and communicating between virtual and physical entities in the Metaverse

Smart mobile sensing for measuring quality of experience (QoE) in urban public transports

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

Behavior Analysis of User Interaction on Online Short Video Platform

Online short video platform (OSVP) can disseminate information in an intuitive and three-dimensional (3D) way, and achieve commercial value in a mobile, socialized, life-oriented manner. To fully understand the information dissemination mode of short videos, it is of practical significance to explore user behaviors and their interactive relationship. Therefore, this paper carries out a behavior analysis on user interaction on OSVP. Firstly, an OSVP user interaction model was established based on three aspects: activity, attention, and interactivity. Next, user preference for short videos was examined through cluster analysis and evaluation. Finally, the authors summarized the behavior features of OSVP user interaction and investigated user interaction to decide which videos should be pushed to users. The average utility and average cost of user interaction behavior were compared, indicators of degree centrality were summarized, and the validity of the proposed algorithm was verified by experimental results