Expressing social attitudes in virtual agents for social training games

The use of virtual agents in social coaching has increased rapidly in the last decade. In order to train the user in different situations than can occur in real life, the virtual agent should be able to express different social attitudes. In this paper, we propose a model of social attitudes that enables a virtual agent to reason on the appropriate social attitude to express during the interaction with a user given the course of the interaction, but also the emotions, mood and personality of the agent. Moreover, the model enables the virtual agent to display its social attitude through its non-verbal behaviour. The proposed model has been developed in the context of job interview simulation. The methodology used to develop such a model combined a theoretical and an empirical approach. Indeed, the model is based both on the literature in Human and Social Sciences on social attitudes but also on the analysis of an audiovisual corpus of job interviews and on post-hoc interviews with the recruiters on their expressed attitudes during the job interview

The Augmented Learner : The pivotal role of multimedia enhanced learning within a foresight-based learning model designed to accelerate the delivery of higher levels of learner creativity

The central theme for this dissertation lies at the intersection of multisensory technology enhanced learning, the field of foresight and transformative pedagogy and their role in helping to develop greater learner creativity. These skills will be key to meeting the needs of the projected growing role of the creative class within the emerging global workforce structure and the projected growth in R&D and the advancement of human-machine resource management. Over the past two decades, we have traversed from the Industrial Age through the Information Age into what we now call postnormal times, manifested partly in Industry 4.0. It is widely considered that the present education system in countries with developed economies is not optimised for delivering the much-needed creative skills, which are prominent amongst the critical 21st C skills required by the creative class, (also known as creatives), which will be increasingly dominant in terms of near future employability. Consequently, there will be a potential shortfall of creatives unless this issue is rapidly addressed. To ensure that the creative skills I aimed to enhance were relevant and aligned with emerging demands of the changing landscape, I deconstructed the critical dimensions, context, and concept of creativity in postnormal times as well as undertaking in-depth research on the potential future workscape and the future of education and learning, applying a comprehensive foresight approach to the latter using a 2030-2040 horizon. Based upon the outcomes of these studies I designed an experimental integrative learning system that I have applied, researched, and evolved over the past 4 years with over 150 students at PhD and master’s level. The system is aimed at generating higher levels of creative engagement and development through a focus on increased immersion and creativity-inducing approaches. The system, which I call the Living Learning System, is based upon eight integrated elements, supported by course development pillars aimed at optimizing learner future skill competencies and levels of creativity for which I apply severalevaluation techniques and metrics. Accordingly, as the central hypothesis of this dissertation, I argue that by integrating the critical elements of the Living Learning System, such as emerging multisensory technology enhanced learning coupled with optimised transformative and experiential learning approaches, framed within the field of foresight, with its futures focus and decentralised thinking approaches, students increase their ability to be creative. This increased ability is based on the student attaining a richer level of personal ambience through deeper immersion generated through higher incidence of self-direction, constructivism-based blended pedagogy, futures literacy, and a balance of decentralised and systems-based thinking, as well as cognitive and social platforms aimed at optimizing learner creative achievement. This dissertation demonstrates how the application of the combined elements of the Living Learning System, with its futures focus and its ensuing transdisciplinary curricula and courses, can provide a clear path towards significantly increased learner creativity. The findings of the quantitative, questionnaire-based research set out in detail in Chapter 9, together with the performance and creativity evaluation models applied against the selected case studies of student projects substantiate the validity of the hypothesis that the application of the Living Learning System with its futures focus leads to increased creativity in line with the needs of the postnormal era.publishedVersio

Affective reactions towards socially interactive agents and their computational modeling

Over the past 30 years, researchers have studied human reactions towards machines applying the Computers Are Social Actors paradigm, which contrasts reactions towards computers with reactions towards humans. The last 30 years have also seen improvements in technology that have led to tremendous changes in computer interfaces and the development of Socially Interactive Agents. This raises the question of how humans react to Socially Interactive Agents. To answer these questions, knowledge from several disciplines is required, which is why this interdisciplinary dissertation is positioned within psychology and computer science. It aims to investigate affective reactions to Socially Interactive Agents and how these can be modeled computationally. Therefore, after a general introduction and background, this thesis first provides an overview of the Socially Interactive Agent system used in this work. Second, it presents a study comparing a human and a virtual job interviewer, which shows that both interviewers induce shame in participants to the same extent. Thirdly, it reports on a study investigating obedience towards Socially Interactive Agents. The results indicate that participants obey human and virtual instructors in similar ways. Furthermore, both types of instructors evoke feelings of stress and shame to the same extent. Fourth, a stress management training using biofeedback with a Socially Interactive Agent is presented. The study shows that a virtual trainer can teach coping techniques for emotionally challenging social situations. Fifth, it introduces MARSSI, a computational model of user affect. The evaluation of the model shows that it is possible to relate sequences of social signals to affective reactions, taking into account emotion regulation processes. Finally, the Deep method is proposed as a starting point for deeper computational modeling of internal emotions. The method combines social signals, verbalized introspection information, context information, and theory-driven knowledge. An exemplary application to the emotion shame and a schematic dynamic Bayesian network for its modeling are illustrated. Overall, this thesis provides evidence that human reactions towards Socially Interactive Agents are very similar to those towards humans, and that it is possible to model these reactions computationally.In den letzten 30 Jahren haben Forschende menschliche Reaktionen auf Maschinen untersucht und dabei das “Computer sind soziale Akteure”-Paradigma genutzt, in dem Reaktionen auf Computer mit denen auf Menschen verglichen werden. In den letzten 30 Jahren hat sich ebenfalls die Technologie weiterentwickelt, was zu einer enormen Veränderung der Computerschnittstellen und der Entwicklung von sozial interaktiven Agenten geführt hat. Dies wirft Fragen zu menschlichen Reaktionen auf sozial interaktive Agenten auf. Um diese Fragen zu beantworten, ist Wissen aus mehreren Disziplinen erforderlich, weshalb diese interdisziplinäre Dissertation innerhalb der Psychologie und Informatik angesiedelt ist. Sie zielt darauf ab, affektive Reaktionen auf sozial interaktive Agenten zu untersuchen und zu erforschen, wie diese computational modelliert werden können. Nach einer allgemeinen Einführung in das Thema gibt diese Arbeit daher, erstens, einen Überblick über das Agentensystem, das in der Arbeit verwendet wird. Zweitens wird eine Studie vorgestellt, in der eine menschliche und eine virtuelle Jobinterviewerin miteinander verglichen werden, wobei sich zeigt, dass beide Interviewerinnen bei den Versuchsteilnehmenden Schamgefühle in gleichem Maße auslösen. Drittens wird eine Studie berichtet, in der Gehorsam gegenüber sozial interaktiven Agenten untersucht wird. Die Ergebnisse deuten darauf hin, dass Versuchsteilnehmende sowohl menschlichen als auch virtuellen Anleiterinnen ähnlich gehorchen. Darüber hinaus werden durch beide Instruktorinnen gleiche Maße von Stress und Scham hervorgerufen. Viertens wird ein Biofeedback-Stressmanagementtraining mit einer sozial interaktiven Agentin vorgestellt. Die Studie zeigt, dass die virtuelle Trainerin Techniken zur Bewältigung von emotional herausfordernden sozialen Situationen vermitteln kann. Fünftens wird MARSSI, ein computergestütztes Modell des Nutzeraffekts, vorgestellt. Die Evaluation des Modells zeigt, dass es möglich ist, Sequenzen von sozialen Signalen mit affektiven Reaktionen unter Berücksichtigung von Emotionsregulationsprozessen in Beziehung zu setzen. Als letztes wird die Deep-Methode als Ausgangspunkt für eine tiefer gehende computergestützte Modellierung von internen Emotionen vorgestellt. Die Methode kombiniert soziale Signale, verbalisierte Introspektion, Kontextinformationen und theoriegeleitetes Wissen. Eine beispielhafte Anwendung auf die Emotion Scham und ein schematisches dynamisches Bayes’sches Netz zu deren Modellierung werden dargestellt. Insgesamt liefert diese Arbeit Hinweise darauf, dass menschliche Reaktionen auf sozial interaktive Agenten den Reaktionen auf Menschen sehr ähnlich sind und dass es möglich ist diese menschlichen Reaktion computational zu modellieren.Deutsche Forschungsgesellschaf

Agents for educational games and simulations

This book consists mainly of revised papers that were presented at the Agents for Educational Games and Simulation (AEGS) workshop held on May 2, 2011, as part of the Autonomous Agents and MultiAgent Systems (AAMAS) conference in Taipei, Taiwan. The 12 full papers presented were carefully reviewed and selected from various submissions. The papers are organized topical sections on middleware applications, dialogues and learning, adaption and convergence, and agent applications

Social VR design features and experiential outcomes:narrative review and relationship map for dyadic agent conversations

The application of virtual reality to the study of conversation and social interaction is a relatively new field of study. While the affordances of VR in the domain compared to traditional methods are promising, the current state of the field is plagued by a lack of methodological standards and shared understanding of how design features of the immersive experience impact participants. In order to address this, this paper develops a relationship map between design features and experiential outcomes, along with expectations for how those features interact with each other. Based on the results of a narrative review drawing from diverse fields, this relationship map focuses on dyadic conversations with agents. The experiential outcomes chosen include presence &amp; engagement, psychological discomfort, and simulator sickness. The relevant design features contained in the framework include scenario agency, visual fidelity, agent automation, environmental context, and audio features. We conclude by discussing the findings of the review and framework, such as the multimodal nature of social VR being highlighted, and the importance of environmental context, and lastly provide recommendations for future research in social VR.</p

Proceedings of the 1993 Conference on Intelligent Computer-Aided Training and Virtual Environment Technology, Volume 1

These proceedings are organized in the same manner as the conference's contributed sessions, with the papers grouped by topic area. These areas are as follows: VE (virtual environment) training for Space Flight, Virtual Environment Hardware, Knowledge Aquisition for ICAT (Intelligent Computer-Aided Training) & VE, Multimedia in ICAT Systems, VE in Training & Education (1 & 2), Virtual Environment Software (1 & 2), Models in ICAT systems, ICAT Commercial Applications, ICAT Architectures & Authoring Systems, ICAT Education & Medical Applications, Assessing VE for Training, VE & Human Systems (1 & 2), ICAT Theory & Natural Language, ICAT Applications in the Military, VE Applications in Engineering, Knowledge Acquisition for ICAT, and ICAT Applications in Aerospace