Analysing the Direction of Emotional Influence in Nonverbal Dyadic Communication: A Facial-Expression Study

Identifying the direction of emotional influence in a dyadic dialogue is of increasing interest in the psychological sciences with applications in psychotherapy, analysis of political interactions, or interpersonal conflict behavior. Facial expressions are widely described as being automatic and thus hard to overtly influence. As such, they are a perfect measure for a better understanding of unintentional behavior cues about social-emotional cognitive processes. With this view, this study is concerned with the analysis of the direction of emotional influence in dyadic dialogue based on facial expressions only. We exploit computer vision capabilities along with causal inference theory for quantitative verification of hypotheses on the direction of emotional influence, i.e., causal effect relationships, in dyadic dialogues. We address two main issues. First, in a dyadic dialogue, emotional influence occurs over transient time intervals and with intensity and direction that are variant over time. To this end, we propose a relevant interval selection approach that we use prior to causal inference to identify those transient intervals where causal inference should be applied. Second, we propose to use fine-grained facial expressions that are present when strong distinct facial emotions are not visible. To specify the direction of influence, we apply the concept of Granger causality to the time series of facial expressions over selected relevant intervals. We tested our approach on newly, experimentally obtained data. Based on the quantitative verification of hypotheses on the direction of emotional influence, we were able to show that the proposed approach is most promising to reveal the causal effect pattern in various instructed interaction conditions.Comment: arXiv admin note: text overlap with arXiv:1810.1217

Structural and effective connectivity reveals potential network-based influences on category-sensitive visual areas

Visual category perception is thought to depend on brain areas that respond specifically when certain categories are viewed. These category-sensitive areas are often assumed to be modules (with some degree of processing autonomy) and to act predominantly on feedforward visual input. This modular view can be complemented by a view that treats brain areas as elements within more complex networks and as influenced by network properties. This network-oriented viewpoint is emerging from studies using either diffusion tensor imaging to map structural connections or effective connectivity analyses to measure how their functional responses influence each other. This literature motivates several hypotheses that predict category-sensitive activity based on network properties. Large, long-range fiber bundles such as inferior fronto-occipital, arcuate and inferior longitudinal fasciculi are associated with behavioural recognition and could play crucial roles in conveying backward influences on visual cortex from anterior temporal and frontal areas. Such backward influences could support top-down functions such as visual search and emotion-based visual modulation. Within visual cortex itself, areas sensitive to different categories appear well-connected (e.g., face areas connect to object- and motion sensitive areas) and their responses can be predicted by backward modulation. Evidence supporting these propositions remains incomplete and underscores the need for better integration of DTI and functional imaging

Knowledge Elicitation Methods for Affect Modelling in Education

Research on the relationship between affect and cognition in Artificial Intelligence in Education (AIEd) brings an important dimension to our understanding of how learning occurs and how it can be facilitated. Emotions are crucial to learning, but their nature, the conditions under which they occur, and their exact impact on learning for different learners in diverse contexts still needs to be mapped out. The study of affect during learning can be challenging, because emotions are subjective, fleeting phenomena that are often difficult for learners to report accurately and for observers to perceive reliably. Context forms an integral part of learners’ affect and the study thereof. This review provides a synthesis of the current knowledge elicitation methods that are used to aid the study of learners’ affect and to inform the design of intelligent technologies for learning. Advantages and disadvantages of the specific methods are discussed along with their respective potential for enhancing research in this area, and issues related to the interpretation of data that emerges as the result of their use. References to related research are also provided together with illustrative examples of where the individual methods have been used in the past. Therefore, this review is intended as a resource for methodological decision making for those who want to study emotions and their antecedents in AIEd contexts, i.e. where the aim is to inform the design and implementation of an intelligent learning environment or to evaluate its use and educational efficacy

Explaining Implicit and Explicit Affective Linkages in IT Teams: Facial Recognition, Emotional Intelligence, and Affective Tone

Over 80 percent of task work in organizations is performed by teams. Most teams operate in a more fluid, dynamic, and complex environment than in the past. As a result, a growing body of research is beginning to focus on how teams’ emotional well-being can benefit the effectiveness of workplace team efforts. These teams are required to be adaptive, to operate in ill-structured environments, and to rely on technology more than ever before. However, teams have become so ubiquitous that many organizations and managers take them for granted and assume they will be effective and productive. Because of the increased use of team work and the lack of sufficient organizational and managerial sufficient best practices for teams, more research is required. Team Emotional Intelligence (TEI) is a collective skill that has been shown to benefit team performance. However, measures for TEI are relatively new and have not been widely studied. Results show TEI is a viable skill that affects performance in IT teams. In technology-rich environments, the teams’ coordination can vary on levels of the expertise needed when TEI behaviors are employed. Cooperative norms play an important role in team interactions and influence TEI. Physiological measures of team emotional contagion and TEI, as well as psychometric measures of team affective tone results show causal affective linkages in the emotional convergence model. These results suggest that combined physiological and psychometric measures of team emotion behavior provide explanatory power for these linkages in teams during IS technology system use. These findings offer new insights into the emotional states of IS teams that may advance the understanding team behaviors for improved performance outcomes and contribute to the NeuroIS literature

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Smell's puzzling discrepancy: Gifted discrimination, yet pitiful identification

Mind &Language, Volume 35, Issue 1, Page 90-114, February 2020

The socio-emotional basis of human interaction and communication: How we construct our social world

A review of dimensional research about (the perception of) feelings, non-verbal and verbal communication, behavior and personality reveals in each domain three very similar dimensions. They originated from diverse research areas, often received different names and are conceptually not identical. Yet, the first dimension seems to share in all five areas a general positive versus negative evaluation (e.g. happiness–disgust or friendliness–hostility), the second a strong versus weak characterization (e.g. anger–fear or dominance–submission) and the third dimension an active versus passive impression (e.g. ecstasy–boredom or high–low arousability). These three dimensions are likely to function as fundamental dimensions of interaction and communication as perceived and enacted by humans of all (investigated) cultures. They are interpreted as a universal socio-emotional space that corresponds to an evolutionary need for coordination between individuals. They are implied in the logic of game, exchange or interdependence theory, and manifest themselves in the cultural meanings predicted by affect control theory. The presented overview and reconstruction combines the largely fragmented views of several diverse research domains into a perspective that fosters interdisciplinary understanding and integrative theory-building about human sociality within and between the social sciences with extensions into the natural sciences and humanities.Un passage en revue de la recherche dimensionnelle sur les sentiments (et leur perception), la communication verbale et non-verbale, le comportement et la personnalité, met en évidence trois dimensions très similaires pour chacun de ces domaines. Elles proviennent de différents domaines de recherche, ont souvent reçu des dénominations différentes et ne sont pas identiques conceptuellement. Cependant, la première dimension semble partager dans ces cinq domaines une évaluation positive versus négative (e.g., joie–dégoût ou amitié–hostilité), la deuxième une caractérisation fort versus faible (e.g. colère–peur ou dominance–soumission) et la troisième une impression actif versus passif (e.g. extase–ennui ou stimulation haute–basse). Ces trois dimensions fonctionnent vraisemblablement comme des dimensions fondamentales d’interaction et de communication perçues et émises par les humains de toutes les cultures (étudiées). Elles sont interprétées comme un espace socio-émotionnel universel qui correspond à un besoin au cours de l’évolution de coordination entre les individus. Elles sont impliquées dans la logique du jeu, de l’échange et la théorie de l’interdépendance, et se manifestent dans les significations culturelles prédites par la théorie du contrôle des affects. La présente étude combine les visions largement fragmentées de nombreux et divers domaines de recherche en une perspective qui veut promouvoir une compréhension interdisciplinaire et construire une théorie intégrative sur la socialité humaine dans et entre les sciences sociales avec des ramifications vers les sciences naturelles et les humanités.Peer Reviewe

The neural architecture of emotional intelligence.

Emotional Intelligence (EI) is a nebulous concept that permeates daily interpersonal communication. Despite prolific research into its benefits, EI subjective measurement is difficult, contributing to an enigmatic definition of its core constructs. However, neuroimaging research probing socioaffective brain mechanisms underlying putative EI constructs can add an objective perspective to existing models, thereby illuminating the nature of EI. Therefore, the primary aim of this dissertation is to identify brain networks underlying EI and examine how EI arises from the brain’s functional and structural neuroarchitecture. EI is first defined according to behavioral data, which suggests EI is made up of two core constructs: Empathy and Emotion Regulation (ER). The interaction of brain networks underlying Empathy and ER is then investigated using a novel neuroimaging analysis method: dynamic functional connectivity (dynFC). The results suggest efficient communication and (re)configuration between the CEN, DMN, SN underlie both ER and RME task dynamics, and that these temporal patterns relate to trait empathy and ER tendency. Given the demonstrated behavioral and neurobiological relationship between empathy and ER, our second aim is to examine each of these constructs individually through detailed experiments using a variety of neuroimaging methodologies. The dissertation concludes by proposing EI is an ability that arises from the effective, yet flexible communication between brain networks underlying Empathy and ER. The dissertation is divided into five chapters. Chapter I describes the foundational concept of EI as originally described by a variety of psychological figures and the lacuna that exists in terms of its neural correlates. Chapter II presents behavioral data that proposes EI is best predicted by Empathy and ER. Chapter III explores the dynamic relationship between brain networks underlying Empathy and ER, with the aim of elucidating their neurobiological associations, and investigate how such associations may combine to create EI. Chapter IV examines Empathy closely, by probing its neurobiological relationship to interoception and anxiety. Chapter V examines ER closely, by investigating whether gender plays a role in ER, and its neurobiological relationship to hormones. Chapter VI links the general findings from Chapters III, IV and V, and proposes an integrative neurocognitive EI model. The dissertation concludes by providing clinical and non-clinical applications for the model