QoE Estimation of WebRTC-based Audio-visual Conversations from Facial and Speech Features

The utilization of user’s facial- and speech-related features for the estimation of the Quality of Experience (QoE) of multimedia services is still underinvestigated despite its potential. Currently, only the use of either facial or speech features individually has been proposed, and relevant limited experiments have been performed. To advance in this respect, in this study, we focused on WebRTC-based videoconferencing, where it is often possible to capture both the facial expressions and vocal speech characteristics of the users. First, we performed thorough statistical analysis to identify the most significant facial- and speech-related features for QoE estimation, which we extracted from the participants’ audio-video data collected during a subjective assessment. Second, we trained individual QoE estimation machine learning-based models on the separated facial and speech datasets. Finally, we employed data fusion techniques to combine the facial and speech datasets into a single dataset to enhance the QoE estimation performance due to the integrated knowledge provided by the fusion of facial and speech features. The obtained results demonstrate that the data fusion technique based on the Improved Centered Kernel Alignment (ICKA) allows for reaching a mean QoE estimation accuracy of 0.93, whereas the values of 0.78 and 0.86 are reached when using only facial or speech features, respectively

Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications.Comment: 28 pages, 14 figure

Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications

Communication systems to date primarily aim at reliably communicating bit sequences. Such an approach provides efficient engineering designs that are agnostic to the meanings of the messages or to the goal that the message exchange aims to achieve. Next generation systems, however, can be potentially enriched by folding message semantics and goals of communication into their design. Further, these systems can be made cognizant of the context in which communication exchange takes place, thereby providing avenues for novel design insights. This tutorial summarizes the efforts to date, starting from its early adaptations, semantic-aware and task-oriented communications, covering the foundations, algorithms and potential implementations. The focus is on approaches that utilize information theory to provide the foundations, as well as the significant role of learning in semantics and task-aware communications

Socialization of Pain Memories:Parent-Child Reminiscing About Past Painful and Sad Events

OBJECTIVE: Parent-child reminiscing about past negative events has been linked to a host of developmental outcomes. Previous research has identified two distinct between-parent reminiscing styles, wherein parents who are more elaborative (vs. repetitive) have children with more optimal outcomes. To date, however, research has not examined how parents and children talk about past painful experiences nor compared parent-child reminiscing about past painful versus other distressing events despite key developmental differences in how young children respond to pain versus sadness in others. This study aimed to fill that gap. METHODS: Seventy-eight children aged 4 to 7 years underwent a tonsillectomy. Two weeks postsurgery, children and one of their parents discussed past autobiographical events (i.e., the tonsillectomy, another painful event, a sad event). Parent-child conversations were coded using established coding schemes to capture parental reminiscing style, content, and autonomy support. RESULTS: Findings revealed robust differences in parent-child reminiscing about painful versus sad events. Parents were less elaborative, used less emotionally negative words and explanations, and were less supportive of their children's autonomy while reminiscing about past painful versus sad events. CONCLUSIONS: These findings demonstrate that through reminiscing, parents may socialize children about pain in a way that is different from other distressing events (e.g., sadness). Future research should examine the influence of differential reminiscing about pain versus sadness on developmental and health outcomes.</p

Socialization of Pain Memories:Parent-Child Reminiscing About Past Painful and Sad Events

Parent-child reminiscing about past negative events has been linked to a host of developmental outcomes. Previous research has identified two distinct between-parent reminiscing styles, wherein parents who are more elaborative (vs. repetitive) have children with more optimal outcomes. To date, however, research has not examined how parents and children talk about past painful experiences nor compared parent-child reminiscing about past painful versus other distressing events despite key developmental differences in how young children respond to pain versus sadness in others. This study aimed to fill that gap

The Listening Shift: Evaluating a Communication-Strategies Training Program for Telepractice Nurses Experiencing Hearing Challenges

Workers who wish to remain employed should be supported in doing so, even if they are experiencing age-related disabilities, such as hearing loss. I aimed to better understand the strategies from which workers with hearing loss might benefit, and how they can be supported in adopting these strategies. To collect rich data, I recruited telepractice nurses who rely on listening to make critical decisions about triaging and health care recommendations. My first research question was: What strategies exist for making telephone speech more intelligible for health care providers and patients with hearing challenges? I performed a scoping review following the Joanna Briggs Institute’s protocol. I identified 11 types of strategies, many of which required cooperation from, and disclosure to, providers’ employers, co-workers, and clients. This led me to consider the public narrative workers associated themselves with when they disclosed. Thus, my second research question was: How do Canadian newspapers portray workers with hearing loss? Through a thematic analysis of newspapers articles on this topic, I found they are predominantly portrayed as striving cheerfully both towards functioning normally and towards differentiating themselves and their hearing loss as unique and positive. To further explore how a subset of adults with hearing loss strive to work with a hearing loss, I developed an online communication-strategies training program tailored to nurses with hearing challenges. I then used a multiple case study to answer the following research question: How do nurses with hearing challenges change in terms of their telephone performance and workplace wellbeing in response to participation in an online communication strategies training program? Results suggested that nurses engaged in a problem-solving process before adopting strategies, and that strategy adoption could positively contribute to their performance. Together, the findings from these studies suggest that strategies exist to enhance the performance of workers with hearing loss, but the process of adopting these strategies can be demanding. Organizations should take steps to proactively support their nurses, health-care providers, and potentially other workers with hearing loss in identifying communication strategies and adapting them to their unique context

Network streaming and compression for mixed reality tele-immersion

Bulterman, D.C.A. [Promotor]Cesar, P.S. [Copromotor

A MODEL FOR PREDICTING THE PERFORMANCE OF IP VIDEOCONFERENCING

With the incorporation of free desktop videoconferencing (DVC) software on the majority of the world's PCs, over the recent years, there has, inevitably, been considerable interest in using DVC over the Internet. The growing popularity of DVC increases the need for multimedia quality assessment. However, the task of predicting the perceived multimedia quality over the Internet Protocol (IP) networks is complicated by the fact that the audio and video streams are susceptible to unique impairments due to the unpredictable nature of IP networks, different types of task scenarios, different levels of complexity, and other related factors. To date, a standard consensus to define the IP media Quality of Service (QoS) has yet to be implemented. The thesis addresses this problem by investigating a new approach to assess the quality of audio, video, and audiovisual overall as perceived in low cost DVC systems. The main aim of the thesis is to investigate current methods used to assess the perceived IP media quality, and then propose a model which will predict the quality of audiovisual experience from prevailing network parameters. This thesis investigates the effects of various traffic conditions, such as, packet loss, jitter, and delay and other factors that may influence end user acceptance, when low cost DVC is used over the Internet. It also investigates the interaction effects between the audio and video media, and the issues involving the lip sychronisation error. The thesis provides the empirical evidence that the subjective mean opinion score (MOS) of the perceived multimedia quality is unaffected by lip synchronisation error in low cost DVC systems. The data-gathering approach that is advocated in this thesis involves both field and laboratory trials to enable the comparisons of results between classroom-based experiments and real-world environments to be made, and to provide actual real-world confirmation of the bench tests. The subjective test method was employed since it has been proven to be more robust and suitable for the research studies, as compared to objective testing techniques. The MOS results, and the number of observations obtained, have enabled a set of criteria to be established that can be used to determine the acceptable QoS for given network conditions and task scenarios. Based upon these comprehensive findings, the final contribution of the thesis is the proposal of a new adaptive architecture method that is intended to enable the performance of IP based DVC of a particular session to be predicted for a given network condition