A genuine smile is indeed in the eyes – The computer aided non-invasive analysis of the exact weight distribution of human smiles across the face

YesUnderstanding the detailed differences between posed and spontaneous smiles is an important topic with a range of applications such as in human-computer interaction, automatic facial emotion analysis and in awareness systems. During the past decade or so, there have been very promising solutions for accurate automatic recognition and detailed facial emotion analysis. To this end, many methods and techniques have been proposed for distinguishing between spontaneous and posed smiles. Our aim here is to go beyond the present state of the art in this field. Hence, in this work, we are concerned with understanding the exact distribution of a smile – both spontaneous and posed – across the face. To do this, we utilise a lightweight computational framework which we have developed to analyse the dynamics of human facial expressions. We utilise this framework to undertake a detailed study of the smile expression. Based on computing the optical flow across the face – especially across key parts of the face such as the mouth, the cheeks and around the eyes – we are able to accurately map the dynamic weight distribution of the smile expression. To validate our computational model, we utilise two publicly available datasets, namely the CK + dataset in which the subjects express posed smiles and the MUG dataset in which the subjects express genuine smiles. Our results not only confirm what already exists in the literature – i.e. that the spontaneous genuine smile is truly in the eyes – but it also gives further insight into the exact distribution of the smile across the face

The computational face for facial emotion analysis: Computer based emotion analysis from the face

Facial expressions are considered to be the most revealing way of understanding the human psychological state during face-to-face communication. It is believed that a more natural interaction between humans and machines can be undertaken through the detailed understanding of the different facial expressions which imitate the manner by which humans communicate with each other. In this research, we study the different aspects of facial emotion detection, analysis and investigate possible hidden identity clues within the facial expressions. We study a deeper aspect of facial expressions whereby we try to identify gender and human identity - which can be considered as a form of emotional biometric - using only the dynamic characteristics of the smile expressions. Further, we present a statistical model for analysing the relationship between facial features and Duchenne (real) and non-Duchenne (posed) smiles. Thus, we identify that the expressions in the eyes contain discriminating features between Duchenne and non-Duchenne smiles. Our results indicate that facial expressions can be identified through facial movement analysis models where we get an accuracy rate of 86% for classifying the six universal facial expressions and 94% for classifying the common 18 facial action units. Further, we successfully identify the gender using only the dynamic characteristics of the smile expression whereby we obtain an 86% classification rate. Likewise, we present a framework to study the possibility of using the smile as a biometric whereby we show that the human smile is unique and stable.Al-Zaytoonah Universit

What’s in a Face? Psychophysiological applications of neuroscience for diagnostics and therapies

The idea that the utility of research should be secondary to understanding its subject delays the extraction of potential value. A parallel translational approach to research was applied whereby discovering new findings and testing their validity was performed in parallel. Research about the face was selected for translation as it provided the complexity, diversity, and fidelity necessary for multiple data-driven hypothesis exploration while remaining key to social interaction. For example, emotional contagion, the tendency for an individual to catch someone else’s emotion has been linked to facial contagion: an automatic reaction whereby facial muscles adopt the expression of any emotional face. Based on the reported exaggerated emotional reactions of patients with upper involvement in Motor Neuron Disease (MND) compared to lower involvement, an experiment was devised to make the difference through comparisons of facial contagion responses with recorded Electromyography (EMG) responses (chapter 3). As these patients were expected to have generally weak responses, it became necessary to increase the sensitivity of acquired signals to elucidate differences between subtypes. An adaptive filtering technique for signal processing was developed based on modelling methods and tested with support vector machines (chapter 2). The therapeutic intervention (chapter 4) consisted of a series of experiments seeking to induce emotional contagion of happiness by presenting images of smiling faces through smartphones. This was also gamified in an experiment at the Science Museum in London to test whether the effect could be found over the short term. Lastly, I parametrised faces from a large population of Tibetan residents and predicted haematological and electrocardiographic measures with machine learning methods as a way of screening for cardiovascular disease through photographs of the face (chapter 5). The results were analysed in relation to the field of cognitive neuroscience and the implications for a parallel translational and high-dimensional approach were discussed

Researching AI Legibility Through Design

Everyday interactions with computers are increasingly likely to involve elements of Artificial Intelligence (AI). Encompassing a broad spectrum of technologies and applications, AI poses many challenges for HCI and design. One such challenge is the need to make AI’s role in a given system legible to the user in a meaningful way. In this paper we employ a Research through Design (RtD) approach to explore how this might be achieved. Building on contemporary concerns and a thorough exploration of related research, our RtD process reflects on designing imagery intended to help increase AI legibility for users. The paper makes three contributions. First, we thoroughly explore prior research in order to critically unpack the AI legibility problem space. Second, we respond with design proposals whose aim is to enhance the legibility, to users, of systems using AI. Third, we explore the role of design-led enquiry as a tool for critically exploring the intersection between HCI and AI research

Giving a voice to the hard to reach: Song as an effective medium for communicating with PMLD children who have low social tolerance

Common practice in special schools is to sing rather than speak to children with Profound and Multiple Learning Difficulties (PMLD), in order to initiate and maintain interactions; however, there is little formal evidence to support this practice. This study explored the extent to which singing is effective with this ‘hard to reach’ cohort. Five pupils participated, who had PMLD and low social tolerance. These individuals do not like to be touched, talked to or to be in close proximity with other people. This research set out to explore the effectiveness of different sorts of interactive approaches, notably singing or speaking, as a starting point for building an evidence base to underpin practice. The study used a single subject research methodology, with an adult as a communication partner who initiated interactions, and responded to the behavioural cues of the child. Interactions were video recorded. A system to code participants’ responses to different interactions was developed, based on detailed descriptions of each individual’s behaviour on three major dimensions: Attention Focus, Social Proximity, and Facial Expression. Pupils’ vocalisations and coordinated actions were also recorded. Events were presented graphically; statistical analyses explored the effectiveness of different interaction approaches; sessions were described qualitatively. The research revealed consistent communicative behaviours (and a means to identify these) in individuals with PMLD and poor social tolerance. Participants were able to express their internal states through consistent patterns in their eye gaze, social proximity, facial expression, and vocal behaviours. The communication partner played a critical role in structuring and directing the interactions; interactions were shaped and influenced by both environment and context. Simple behavioural descriptors are insensitive to context. The research showed that, used on their own, they can lead to misinterpretations of events, and so must be complemented by qualitative descriptions. Nevertheless, the microanalysis of behaviours revealed ‘moments of wonder’ which overturned expectations about who was leading interactions; none of the participants was thought (by staff) to be capable of the secondary intersubjectivity and attention directing behaviours that were documented. There were individual differences in response to singing: however, overall, singing was associated with more positive facial expressions (smiles), higher levels of social tolerance (to touch and proximity), and improved communicative responses (eye contact, vocalisations, and coordinated actions). This provides evidence to support current practices of singing to children with PMLD. This thesis modelled a strategy for collating a profile of communicative behaviours. A practical outcome of the research was that details of the communicative behaviours of participants were circulated via a ‘communication passport’ for each participant, and shared with parents and carers. The use of song became a more explicit part of the daily routine and a planned element in lessons. Findings are related to research on mother-infant communication and infant development, and on the role of music in emotional regulation, and the psychology of music. Directions for future research are discussed

Affective Computing

This book provides an overview of state of the art research in Affective Computing. It presents new ideas, original results and practical experiences in this increasingly important research field. The book consists of 23 chapters categorized into four sections. Since one of the most important means of human communication is facial expression, the first section of this book (Chapters 1 to 7) presents a research on synthesis and recognition of facial expressions. Given that we not only use the face but also body movements to express ourselves, in the second section (Chapters 8 to 11) we present a research on perception and generation of emotional expressions by using full-body motions. The third section of the book (Chapters 12 to 16) presents computational models on emotion, as well as findings from neuroscience research. In the last section of the book (Chapters 17 to 22) we present applications related to affective computing

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

Social and Affective Neuroscience of Everyday Human Interaction

This Open Access book presents the current state of the art knowledge on social and affective neuroscience based on empirical findings. This volume is divided into several sections first guiding the reader through important theoretical topics within affective neuroscience, social neuroscience and moral emotions, and clinical neuroscience. Each chapter addresses everyday social interactions and various aspects of social interactions from a different angle taking the reader on a diverse journey. The last section of the book is of methodological nature. Basic information is presented for the reader to learn about common methodologies used in neuroscience alongside advanced input to deepen the understanding and usability of these methods in social and affective neuroscience for more experienced readers

Non-invasive aesthetic medical tourism in Bangkok: a qualitative analysis of stakeholder risk

Medical tourism is a niche market. Non-invasive aesthetic medical tourism is a type of cosmetic medical tourism with no surgery involved. This type of global interconnection is a relatively new phenomenon within the context of the current sphere of internationalism, thus explaining why few have explored the economic and health factors in a deeper context than simply a cost/ benefit analysis. Therefore, the purpose of this research is to investigate how risk of stakeholders may act as a barrier to the development of aesthetic medical tourism, and to establish how competitive advantage may be sustained in Bangkok. Semi-structured interviews with 15 non-invasive aesthetic clinic owners, 25 English-speaking international tourist-patients, and five government representatives have been conducted in Bangkok. The findings from fieldwork were coded and analysed thematically using a framework derived from the literature review. This research identified six factors related to Thailand's competitiveness and six barriers to the development of this industry. The research made a significant original contribution to academic and practitioner knowledge in that it examined and evaluated risk perception in a new tourism context and with a new group of tourists. This research has established a classification of six types of risk in relation to non-invasive aesthetic medical tourism: functional, physical, financial, time, psychological and social risk. It also demonstrated how interpretivist qualitative approach can make a contribution to aesthetic medical tourism research practice. A framework of risks in relation to the development and management of aesthetic medical tourism in Bangkok was also established for both consumers and service providers in order to realise related risks and develop risk reduction strategies appropriately