Evaluation of animation and lip-sync of avatars, and user interaction in immersive virtual reality learning environments

Virtual Reality (VR) has been showing potential in new and diverse areas, notably in education. However, there is a lack of studies in the Foreign Language Teaching and Learning field, particularly in listening comprehension. Therefore, this study investigated the effects of avatar animations and lip synchronization, and user interaction; features deemed relevant in this broader area. A sociodemographic, a quick CEFR - Common European Framework of Reference for Languages - 15-minute English test, and questionnaire were used to evaluate the participants’ Presence, Quality of Experience, Cybersickness and Knowledge Retention. Results show that, overall, the use of avatars with realistic animations and movements, and featuring lip synchronization have a positive influence on the users’ sense of presence, knowledge retention and a more enjoyable overall quality of experience. The same can be said for the use of object interaction and navigation in the cultural representative environment, which had an overall positive impact.This work is co-financed by the ERDF – European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation - COMPETE 2020 under the PORTUGAL 2020 Partnership Agreement, and through the Portuguese National Innovation Agency (ANI) as a part of project “SMARTCUT - Diagnóstico e Manutenção Remota e Simuladores para Formação de operação e manutenção de Máquinas Florestais: POCI-01-0247-FEDER-048183”.info:eu-repo/semantics/publishedVersio

The mobile information access experience - A user perspective

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Mobile technologies, such as mobile phones, smartphones and Palmtop computers, are in an upwards trend and earliest models of such devices are already available to end-users to communicate and access multimedia content on-the-move. As a logical outcome of this development in mobile technologies and devices, content provider companies have already started investing and piloting mobile multimedia content distribution and broadcasting technologies. Nevertheless, no matter how cutting-edge technology is and no matter how stylish the mobile devices are, the ultimate success of wireless communication technologies and devices are directly associated with the user adoption and embrace of these new equipment and technologies. In this perspective, since multimedia content, for mobile or not, is ultimately produced for the education and/or enjoyment of viewers, the user's perspective concerning the presentation quality is surely of equal importance as objective Quality of Service (QoS) technical parameters, to defining distributed multimedia quality. In order to comprehensively understand user experiences whilst accessing information using mobile devices and technologies, we investigate user-mobile device interaction and look into the surrounding issues in a uniform manner by combining multiple aspects: user initial device experience (Out-of-Box Experience), mobile information access in a real-world context, device impact on user information access and perceptually tailored multimedia content impact on user information assimilation and satisfaction. Accordingly, an extensive experimental investigation has been undertaken to see how user experiences varied based on device familiarity, device type, real-world context and variable locations. The findings has shown that the overall perception, and effectively the user information access experience, is affected and improved when multimedia content is tailored according to user device type and context. Thus highlights that the future of mobile computing necessitates two-faceted research, which should combine both a user as well as a technical perspective

Distributed multimedia quality: The user perspective

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Distributed multimedia supports a symbiotic infotainment duality, i.e. the ability to transfer information to the user, yet also provide the user with a level of satisfaction. As multimedia is ultimately produced for the education and / or enjoyment of viewers, the user’s-perspective concerning the presentation quality is surely of equal importance as objective Quality of Service (QoS) technical parameters, to defining distributed multimedia quality. In order to extensively measure the user-perspective of multimedia video quality, we introduce an extended model of distributed multimedia quality that segregates quality into three discrete levels: the network-level, the media-level and content-level, using two distinct quality perspectives: the user-perspective and the technical-perspective. Since experimental questionnaires do not provide continuous monitoring of user attention, eye tracking was used in our study in order to provide a better understanding of the role that the human element plays in the reception, analysis and synthesis of multimedia data. Results showed that video content adaptation, results in disparity in user video eye-paths when: i) no single / obvious point of focus exists; or ii) when the point of attention changes dramatically. Accordingly, appropriate technical- and user-perspective parameter adaptation is implemented, for all quality abstractions of our model, i.e. network-level (via simulated delay and jitter), media-level (via a technical- and user-perspective manipulated region-of-interest attentive display) and content-level (via display-type and video clip-type). Our work has shown that user perception of distributed multimedia quality cannot be achieved by means of purely technical-perspective QoS parameter adaptation

Final Report to NSF of the Standards for Facial Animation Workshop

The human face is an important and complex communication channel. It is a very familiar and sensitive object of human perception. The facial animation field has increased greatly in the past few years as fast computer graphics workstations have made the modeling and real-time animation of hundreds of thousands of polygons affordable and almost commonplace. Many applications have been developed such as teleconferencing, surgery, information assistance systems, games, and entertainment. To solve these different problems, different approaches for both animation control and modeling have been developed

Enhancing musical experience for the hearing - impaired using visual and haptic displays

Ph.DDOCTOR OF PHILOSOPH

Coordination of Nods in Dialogue.

PhD ThesisBehavioral mimicry has been claimed to be a nonconscious behavior that evokes prosocial e ects | liking, trust, empathy, persuasiveness | between interaction partners. Recently Intelligent Virtual Agents (IVAs) and Immersive Virtual Environments (IVEs) have provided rich new possibilities for nonverbal behavior studies such as mimicry studies. One of the best known e ects is the \Digital Chameleons" in which an IVA appears to be more persuasive if it automatically mimics a listener's head nods. However, this e ect has not been consistently replicated. This thesis explores the basis of the \chameleon e ects" using a customized IVE integrated with full-body motion capture system that support realtime behavior manipulation in the IVE. Two replications exploring the e ectiveness of the virtual speaker and head nodding behavior of interaction partners in the agent-listener interaction and avatar-listener interaction by manipulating the virtual speaker's head nods and provide mixed results. The rst experiment fails to replicate the original nding of mimicry leading to higher ratings of an agent's e ectiveness. The second experiment shows a higher rating for agreement with a mimicking avatar. Overall, an avatar speaker appears more likely to activate an e ect of behavioral mimicry than an agent speaker, probably because the avatar speaker provides richer nonverbal cues than the agent speaker. Detailed analysis of the motion data for speaker and listener head movements reveals systematic di erences in a) head nodding between a speaker producing a monologue and a speaker engaged in a dialogue b) head nodding of speakers and listeners in the high and low frequency domain and c) the reciprocal dynamics of head-nodding with di erent virtual speaker's head nodding behavior. We conclude that: i) the activation of behavioral mimicry requires a certain number of nonverbal cues, ii) speakers behave di erently in monologue and dialogue, iii) speakers and listeners nod asymmetrically in di erent frequency domains, iv) the coordination of head nods in natural dialogue is no more than we would expect by chance, v) speakers' and listeners' head nods become coordinated by spontaneous collaborative adjustment of their head nods

Experimental studies of the interaction between people and virtual humans with a focus on social anxiety

Psychotherapy has been one of the major applications of Virtual Reality technology; examples include fear of flying, heights, spiders, and post‐traumatic stress disorder. Virtual reality has been shown to be useful, in the context of exposure therapy for the treatment of social anxiety, such as fear of public speaking, where the clients learn how to conquer their anxiety through interactions with Virtual Characters (avatars). This thesis is concerned with the interaction between human participants and avatars in a Virtual Environment (VE), with the main focus being on Social Anxiety. It is our hypothesis that interactions between people and avatars can evoke in people behaviours that correspond to their degree of social anxiety or confidence. Moreover the responses of people to avatars will also depend on their degree of exhibited social anxiety – they will react differently to a shy avatar compared to a confident avatar. The research started with an experimental study on the reaction of shy and confident male volunteers to an approach by an attractive and friendly virtual woman in a VE. The results show that the participants responded according to expectations towards the avatar at an emotional, physiological, and behavioural level. The research then studied a particular cue which represents shyness – “blushing”. Experiments were carried out on how participant responds towards a blushing avatar. The results suggested that, even without consciously noticing the avatar’s blushing, the participants had an improved relationship with her when she was blushing. Finally, the research further investigated how people respond towards a shy avatar as opposed to a confident one. The results show that participants gave more positive comments to the personality of the avatar displaying signs of shyness

Proceedings of the 6th international conference on disability, virtual reality and associated technologies (ICDVRAT 2006)

The proceedings of the conferenc

A MODEL FOR PREDICTING THE PERFORMANCE OF IP VIDEOCONFERENCING

With the incorporation of free desktop videoconferencing (DVC) software on the majority of the world's PCs, over the recent years, there has, inevitably, been considerable interest in using DVC over the Internet. The growing popularity of DVC increases the need for multimedia quality assessment. However, the task of predicting the perceived multimedia quality over the Internet Protocol (IP) networks is complicated by the fact that the audio and video streams are susceptible to unique impairments due to the unpredictable nature of IP networks, different types of task scenarios, different levels of complexity, and other related factors. To date, a standard consensus to define the IP media Quality of Service (QoS) has yet to be implemented. The thesis addresses this problem by investigating a new approach to assess the quality of audio, video, and audiovisual overall as perceived in low cost DVC systems. The main aim of the thesis is to investigate current methods used to assess the perceived IP media quality, and then propose a model which will predict the quality of audiovisual experience from prevailing network parameters. This thesis investigates the effects of various traffic conditions, such as, packet loss, jitter, and delay and other factors that may influence end user acceptance, when low cost DVC is used over the Internet. It also investigates the interaction effects between the audio and video media, and the issues involving the lip sychronisation error. The thesis provides the empirical evidence that the subjective mean opinion score (MOS) of the perceived multimedia quality is unaffected by lip synchronisation error in low cost DVC systems. The data-gathering approach that is advocated in this thesis involves both field and laboratory trials to enable the comparisons of results between classroom-based experiments and real-world environments to be made, and to provide actual real-world confirmation of the bench tests. The subjective test method was employed since it has been proven to be more robust and suitable for the research studies, as compared to objective testing techniques. The MOS results, and the number of observations obtained, have enabled a set of criteria to be established that can be used to determine the acceptable QoS for given network conditions and task scenarios. Based upon these comprehensive findings, the final contribution of the thesis is the proposal of a new adaptive architecture method that is intended to enable the performance of IP based DVC of a particular session to be predicted for a given network condition