Investigating Context Awareness of Affective Computing Systems: A Critical Approach

AbstractIntelligent Human Computer Interaction systems should be affective aware and Affective Computing systems should be context aware. Positioned in the cross-section of the research areas of Interaction Context and Affective Computing current paper investigates if and how context is incorporated in automatic analysis of human affective behavior. Several related aspects are discussed ranging from modeling, acquiring and annotating issues in affectively enhanced corpora to issues related to incorporating context information in a multimodal fusion framework of affective analysis. These aspects are critically discussed in terms of the challenges they comprise while, in a wider framework, future directions of this recently active, yet mainly unexplored, research area are identified. Overall, the paper aims to both document the present status as well as comment on the evolution of the upcoming topic of Context in Affective Computing

Multi-channel Transformers for Multi-articulatory Sign Language Translation

Sign languages use multiple asynchronous information channels (articulators), not just the hands but also the face and body, which computational approaches often ignore. In this paper we tackle the multi-articulatory sign language translation task and propose a novel multi-channel transformer architecture. The proposed architecture allows both the inter and intra contextual relationships between different sign articulators to be modelled within the transformer network itself, while also maintaining channel specific information. We evaluate our approach on the RWTH-PHOENIX-Weather-2014T dataset and report competitive translation performance. Importantly, we overcome the reliance on gloss annotations which underpin other state-of-the-art approaches, thereby removing future need for expensive curated datasets

Adaptive Gesture Recognition with Variation Estimation for Interactive Systems

This paper presents a gesture recognition/adaptation system for Human Computer Interaction applications that goes beyond activity classification and that, complementary to gesture labeling, characterizes the movement execution. We describe a template-based recognition method that simultaneously aligns the input gesture to the templates using a Sequential Montecarlo inference technique. Contrary to standard template- based methods based on dynamic programming, such as Dynamic Time Warping, the algorithm has an adaptation process that tracks gesture variation in real-time. The method continuously updates, during execution of the gesture, the estimated parameters and recognition results which offers key advantages for continuous human-machine interaction. The technique is evaluated in several different ways: recognition and early recognition are evaluated on a 2D onscreen pen gestures; adaptation is assessed on synthetic data; and both early recognition and adaptation is evaluation in a user study involving 3D free space gestures. The method is not only robust to noise and successfully adapts to parameter variation but also performs recognition as well or better than non-adapting offline template-based methods

Virtual character animation: affective analysis in human computer interaction

This thesis deals with the field of Affective Computing and Human Computer Interaction. Chapter 2 presents new aspects of affective computing, namely multimodal, dynamic emotion recognition focusing on naturalistic behavior and context adaptation, while chapter 3 deals with the computational formalization of gesture expressivity features, with validation of the extraction algorithm and with the derivation of a framework for multimodal and expressive synthesis on Embodied Conversational Agents. Chapter 4 deals with aspects of human computer interaction such as alternative means of interaction, Universal Access and Assistive Technology by proposing a novel scheme for automatic gesture and sign language recognition and a sign language synthesis platform. Furthermore, gesture recognition contributes to emotion analysis both by extracting primary information concerning the emotional content that accompanies a gesture and by supporting the qualitative process of expressivity features extraction.Η παρούσα διδακτορική διατριβή κινείται στο πλαίσιο της συναισθηματικής υπολογιστικής (affective computing) και της αλληλεπίδρασης ανθρώπου μηχανής (human computer interaction). Τα κεφάλαια 2 και 3 αφορούν στην συναισθηματική υπολογιστική και ειδικότερα: α) στην αναγνώριση δυναμικών συναισθηματικών καταστάσεων από πολλαπλές μορφές πληροφορίας κατά την φυσική επικοινωνία ανθρώπου μηχανής αλλά και στην διαδικασία εντοπισμού και προσαρμογής σε δυναμικές συνθήκες και αλλαγές του εννοιολογικού πλαισίου της αλληλεπίδρασης και β) στην τυποποίηση εκφραστικώνπαραμέτρων χειρονομιών κατά την συναισθηματικά εμπλουτισμένη αλληλεπίδραση και στην αξιολόγηση της τυποποίησης αυτής μέσω εικονικών χαρακτήρων από ανθρώπινους χρήστες. Το κεφάλαιο 4 αναφέρεται στην αναγνώριση χειρονομιών ως μέσο αλληλεπίδρασης εναλλακτικό των καθιερωμένων, αλλά και στην αναγνώριση και σύνθεση νοηματικής γλώσσας (ενότητες 4.2 και 4.3 αντίστοιχα) συνεισφέροντας στις έννοιες της καθολικής πρόσβασης (universal access) και υποστηρικτικής τεχνολογίας (assistive technology). Η αναγνώριση χειρονομιών αφορά και στην συναισθηματική υπολογιστική εξάγοντας πληροφορίες σχετικά με την έννοια και το συναισθηματικό περιεχόμενο κάποιας χειρονομίας αλλά και υποβοηθώντας την εξαγωγή ποιοτικών, συναισθηματικών εκφραστικών παραμέτρων

A Survey on Computational and Emergent Digital Storytelling

The research field of digital storytelling is cross-disciplinary and extremely wide. In this paper, methods, frameworks, and tools that have been created for authoring and presenting digital narratives, are selected and examined among hundreds of works. The basic criterion for selecting these works has been their ability to create content by computational, emergent methods. By delving into the work of many researchers, the objective is to study current trends in this research field and discuss possible future directions. Most of the relevant tools and methods have been designed with a specific purpose in mind, but their use could be expanded to other areas of interest or could at least be the steppingstone for other ideas. Therefore, the following works show elements of computational and emergent narrative creation and a classification is proposed according to their purpose of existence. Finally, new potential research directions in the field are identified and possible future research steps are discussed