Emotional Biosensing: Exploring Critical Alternatives

Emotional biosensing is rising in daily life: Data and categories claim to know how people feel and suggest what they should do about it, while CSCW explores new biosensing possibilities. Prevalent approaches to emotional biosensing are too limited, focusing on the individual, optimization, and normative categorization. Conceptual shifts can help explore alternatives: toward materiality, from representation toward performativity, inter-action to intra-action, shifting biopolitics, and shifting affect/desire. We contribute (1) synthesizing wide-ranging conceptual lenses, providing analysis connecting them to emotional biosensing design, (2) analyzing selected design exemplars to apply these lenses to design research, and (3) offering our own recommendations for designers and design researchers. In particular we suggest humility in knowledge claims with emotional biosensing, prioritizing care and affirmation over self- improvement, and exploring alternative desires. We call for critically questioning and generatively re- imagining the role of data in configuring sensing, feeling, ‘the good life,’ and everyday experience

The behavioural impact of a visually represented virtual assistant in a selfservice checkout context

Our research investigated whether the presence of an interface agent - or virtual assistant (VA) - in a self-service checkout context has behavioural effects on the transaction process during particular tasks. While many participants claimed to have not noticed a VA within the self-service interface, behaviour was still affected, i.e. fewer people made errors with the VA present than in the voice-only and control conditions. The results are explained as reflective of an unconscious observation of non-verbal cues exhibited by the VA. The results are discussed in relation to possible behavioural outcomes of VA presence.</p

Virtual coaches for healthy lifestyle

Since the introduction of the idea of the software interface agent the question recurs whether these agents should be personified and graphically visualized in the interface. In this chapter we look at the use of virtual humans in the interface of healthy lifestyle coaching systems. Based on theory of persuasive communication we analyse the impact that the use of graphical interface agents may have on user experience and on the efficacy of this type of persuasive systems. We argue that research on the impact of a virtual human interface on the efficacy of these systems requires longitudinal field studies in addition to the controlled short-term user evaluations in the field of human computer interaction (HCI). We introduce Kristina, a mobile personal coaching system that monitors its user’s physical activity and that presents feedback messages to the user. We present results of field trials (N = 60, 7 weeks) in which we compare two interface conditions on a smartphone. In one condition feedback messages are presented by a virtual animated human, in the other condition they are displayed on the screen in text. Results of the field trials show that user motivation, use context and the type of device on which the feedback message is received influence the perception of the presentation format of feedback messages and the effect on compliance to the coaching regime

Comparing Photorealistic and Animated Embodied Conversational Agents in Serious Games: An Empirical Study on User Experience

Embodied conversational agents (ECAs) are paradigms of conversational user interfaces in the form of embodied characters. While ECAs offer various manipulable features, this paper focuses on a study conducted to explore two distinct levels of presentation realism. The two agent versions are photorealistic and animated. The study aims to provide insights and design suggestions for speech-enabled ECAs within serious game environments. A within-subjects, two-by-two factorial design was employed for this research with a cohort of 36 participants balanced for gender. The results showed that both the photorealistic and the animated versions were perceived as highly usable, with overall mean scores of 5.76 and 5.71, respectively. However, 69.4 per cent of the participants stated they preferred the photorealistic version, 25 per cent stated they preferred the animated version and 5.6 per cent had no stated preference. The photorealistic agents were perceived as more realistic and human-like, while the animated characters made the task feel more like a game. Even though the agents' realism had no significant effect on usability, it positively influenced participants' perceptions of the agent. This research aims to lay the groundwork for future studies on ECA realism's impact in serious games across diverse contexts.Comment: 21 pages, 14 figures, preprint to be published in HCI INTERNATIONAL 2023 25TH INTERNATIONAL CONFERENCE ON HUMAN-COMPUTER INTERACTION proceeding

Cultural dialects of real and synthetic emotional facial expressions

In this article we discuss the aspects of designing facial expressions for virtual humans (VHs) with a specific culture. First we explore the notion of cultures and its relevance for applications with a VH. Then we give a general scheme of designing emotional facial expressions, and identify the stages where a human is involved, either as a real person with some specific role, or as a VH displaying facial expressions. We discuss how the display and the emotional meaning of facial expressions may be measured in objective ways, and how the culture of displayers and the judges may influence the process of analyzing human facial expressions and evaluating synthesized ones. We review psychological experiments on cross-cultural perception of emotional facial expressions. By identifying the culturally critical issues of data collection and interpretation with both real and VHs, we aim at providing a methodological reference and inspiration for further research

Avatars and computer-mediated communication: a review of the definitions, uses, and effects of digital representations

Avatars are growing in popularity and present in many interfaces used for computer-mediated communication (CMC) including social media, e-commerce, and education. Communication researchers have been investigating avatars for over twenty years, and an examination of this literature reveals similarities but also notable discrepancies in conceptual definitions. The goal of this review is to provide a general overview of current debates, methodological approaches, and trends in findings. Our review synthesizes previous research in four areas. First, we examine how scholars have conceptualized the term “avatar,” identify similarities and differences across these definitions, and recommend that scholars use the term consistently. Next, we review theoretical perspectives relevant to avatar perception (e.g., the computers as social actors framework). Then, we examine avatar characteristics that communicators use to discern the humanity and social potential of an avatar (anthropomorphism, form realism, behavioral realism, and perceived agency) and discuss implications for attributions and communication outcomes. We also review findings on the social categorization of avatars, such as when people apply categories like sex, gender, race, and ethnicity to their evaluations of digital representations. Finally, we examine research on avatar selection and design relevant to communication outcomes. Here, we review both motivations in CMC contexts (such as self-presentation and identity expression) and potential effects (e.g., persuasion). We conclude with a discussion of future directions for avatar research and propose that communication researchers consider avatars not just as a topic of study, but also as a tool for testing theories and understanding critical elements of human communication. Avatar mediated environments provide researchers with a number of advantageous technological affordances that can enable manipulations that may be difficult or inadvisable to execute in natural environments. We conclude by discussing the use of avatar research to extend communication theory and our understanding of communication processes

Interactive Virtual Training: Implementation for Early Career Teachers to Practice Classroom Behavior Management

Teachers that are equipped with the skills to manage and prevent disruptive behaviors increase the potential for their students to achieve academically and socially. Student success increases when prevention strategies and effective classroom behavior management (CBM) are implemented in the classroom. However, teachers with less than 5 years of experience, early career teachers (ECTs), are ill equipped to handle disruptive students. ECTs describe disruptive behaviors as a major factor for stress given their limited training in CBM. As a result, disruptive behaviors are reported by ECTs as one of the main reasons for leaving the field. Virtual training environments (VTEs) combined with advances in virtual social agents can support the training of CBM. Although VTEs for teachers already exist, requirements to guide future research and development of similar training systems have not been defined. We propose a set of six requirements for VTEs for teachers. Our requirements were established from a survey of the literature and from iterative lifecycle activities to build our own VTE for teachers. We present different evaluations of our VTE using methodologies and metrics we developed to assess whether all requirements were met. Our VTE simulates interactions with virtual animated students based on real classroom situations to help ECTs practice their CBM. We enhanced our classroom simulator to further explore two aspects of our requirements: interaction devices and emotional virtual agents. Interactions devices were explored by comparing the effect of immersive technologies on users\u27 experience (UX) such as presence, co-presence, engagement and believability. We adapted our VTE originally built for desktop computer, to be compatible with two immersive VR platforms. Results show that our VTE generates high levels of UX across all VR platforms. Furthermore, we enhanced our virtual students to display emotions using facial expressions as current studies do not address whether emotional virtual agents provide the same level of UX across different VR platforms. We assessed the effects of VR platforms and display of emotions on UX. Our analysis shows that facial expressions have greater impact when using a desktop computer. We propose future work on immersive VTEs using emotional virtual agents

Dynamic Facial Expression of Emotion Made Easy

Facial emotion expression for virtual characters is used in a wide variety of areas. Often, the primary reason to use emotion expression is not to study emotion expression generation per se, but to use emotion expression in an application or research project. What is then needed is an easy to use and flexible, but also validated mechanism to do so. In this report we present such a mechanism. It enables developers to build virtual characters with dynamic affective facial expressions. The mechanism is based on Facial Action Coding. It is easy to implement, and code is available for download. To show the validity of the expressions generated with the mechanism we tested the recognition accuracy for 6 basic emotions (joy, anger, sadness, surprise, disgust, fear) and 4 blend emotions (enthusiastic, furious, frustrated, and evil). Additionally we investigated the effect of VC distance (z-coordinate), the effect of the VC's face morphology (male vs. female), the effect of a lateral versus a frontal presentation of the expression, and the effect of intensity of the expression. Participants (n=19, Western and Asian subjects) rated the intensity of each expression for each condition (within subject setup) in a non forced choice manner. All of the basic emotions were uniquely perceived as such. Further, the blends and confusion details of basic emotions are compatible with findings in psychology

How the Brain Makes Up the Mind: a heuristic approach to the hard problem of consciousness

A solution to the “hard problem” requires taking the point of view of the organism and its sub- agents. The organism constructs phenomenality through acts of fiat, much as we create meaning in language, through the use of symbols that are assigned meaning in the context of an embodied evolutionary history. Phenomenality is a virtual representation, made to itself by an executive agent (the conscious self), which is tasked with monitoring the state of the organism and its environment, planning future action, and coordinating various sub-agencies. Consciousness is not epiphenomenal and serves a function for higher organisms that is distinct from unconscious processing. While a strictly scientific solution to the hard problem is not possible for a science that excludes the subjectivity it seeks to explain, there is hope to at least informally bridge the explanatory gulf between mind and matter