The Body Action and Posture Coding System (BAP): Development and Reliability

Several methods are available for coding body movement in nonverbal behavior research, but there is no consensus on a reliable coding system that can be used for the study of emotion expression. Adopting an integrative approach, we developed a new method, the body action and posture coding system, for the time-aligned micro description of body movement on an anatomical level (different articulations of body parts), a form level (direction and orientation of movement), and a functional level (communicative and self-regulatory functions). We applied the system to a new corpus of acted emotion portrayals, examined its comprehensiveness and demonstrated intercoder reliability at three levels: (a) occurrence, (b) temporal precision, and (c) segmentation. We discuss issues for further validation and propose some research application

Nonverbal Social Sensing: What Social Sensing Can and Cannot Do for the Study of Nonverbal Behavior From Video

The study of nonverbal behavior (NVB), and in particular kinesics (i.e., face and body motions), is typically seen as cost-intensive. However, the development of new technologies (e.g., ubiquitous sensing, computer vision, and algorithms) and approaches to study social behavior [i.e., social signal processing (SSP)] makes it possible to train algorithms to automatically code NVB, from action/motion units to inferences. Nonverbal social sensing refers to the use of these technologies and approaches for the study of kinesics based on video recordings. Nonverbal social sensing appears as an inspiring and encouraging approach to study NVB at reduced costs, making it a more attractive research field. However, does this promise hold? After presenting what nonverbal social sensing is and can do, we discussed the key challenges that researchers face when using nonverbal social sensing on video data. Although nonverbal social sensing is a promising tool, researchers need to be aware of the fact that algorithms might be as biased as humans when extracting NVB or that the automated NVB coding might remain context-dependent. We provided study examples to discuss these challenges and point to potential solutions

Pink for girls, red for boys, and blue for both genders: Colour preferences in children and adults

Colours carry social connotations like pink for girls and blue for boys. In a cross-sectional study, we investigated whether such early gender coding might be reflected in absolute colour preferences in children and adults of both genders. In two studies, participants selected their favourite (and least favourite, Study 2) colour from an unrestricted sample of colours. We tested 129 Swiss children (Study 1, 10–14 years-old, 68 boys) and 180 Swiss adults (Study 2, 17–48 years-old, 88 men). In children, we observed that girls chose pink/purple as their favourite hue more often than boys did, the most common favourite hue in girls and boys was blue, and boys chose red as their favourite more often than girls did. In adults, we observed that both genders almost never choose pink as their favourite, blue was a common favourite colour, and women were more likely to favour red than were men. In an additional study (n = 183 Swiss participants, 47 men), we tested whether liking of pink, blue, and red was related to emotion associations with these colours. Pink was associated with positive emotions to the same extent as blue and red. Women further associated more positive emotions with pink than did men. We conclude that some commonalities (blue) and gender differences (pink and red) exist in absolute colour preferences. These differences, however, cannot be fully accounted by emotional associations. We speculate about these gendered colour preferences in relation to gender stereotypes and status differences between men and women

A machine learning approach to quantify the specificity of colour-emotion associations and their cultural differences

The link between colour and emotion and its possible similarity across cultures are questions that have not been fully resolved. Online, 711 participants from China, Germany, Greece and the UK associated 12 colour terms with 20 discrete emotion terms in their native languages. We propose a machine learning approach to quantify (a) the consistency and specificity of colour–emotion associations and (b) the degree to which they are country-specific, on the basis of the accuracy of a statistical classifier in (a) decoding the colour term evaluated on a given trial from the 20 ratings of colour–emotion associations and (b) predicting the country of origin from the 240 individual colour–emotion associations, respectively. The classifier accuracies were significantly above chance level, demonstrating that emotion associations are to some extent colour-specific and that colour–emotion associations are to some extent country-specific. A second measure of country-specificity, the in-group advantage of the colour-decoding accuracy, was detectable but relatively small (6.1%), indicating that colour–emotion associations are both universal and culture-specific. Our results show that machine learning is a promising tool when analysing complex datasets from emotion research

Universal Patterns in Color-Emotion Associations Are Further Shaped by Linguistic and Geographic Proximity

Many of us "see red," "feel blue," or "turn green with envy." Are such color-emotion associations fundamental to our shared cognitive architecture, or are they cultural creations learned through our languages and traditions? To answer these questions, we tested emotional associations of colors in 4,598 participants from 30 nations speaking 22 native languages. Participants associated 20 emotion concepts with 12 color terms. Pattern-similarity analyses revealed universal color-emotion associations (average similarity coefficientr= .88). However, local differences were also apparent. A machine-learning algorithm revealed that nation predicted color-emotion associations above and beyond those observed universally. Similarity was greater when nations were linguistically or geographically close. This study highlights robust universal color-emotion associations, further modulated by linguistic and geographic factors. These results pose further theoretical and empirical questions about the affective properties of color and may inform practice in applied domains, such as well-being and design.Peer reviewe

The sun is no fun without rain : Physical environments affect how we feel about yellow across 55 countries

Across cultures, people associate colours with emotions. Here, we test the hypothesis that one driver of this cross-modal correspondence is the physical environment we live in. We focus on a prime example – the association of yellow with joy, – which conceivably arises because yellow is reminiscent of life-sustaining sunshine and pleasant weather. If so, this association should be especially strong in countries where sunny weather is a rare occurrence. We analysed yellow-joy associations of 6625 participants from 55 countries to investigate how yellow-joy associations varied geographically, climatologically, and seasonally. We assessed the distance to the equator, sunshine, precipitation, and daytime hours. Consistent with our hypotheses, participants who live further away from the equator and in rainier countries are more likely to associate yellow with joy. We did not find associations with seasonal variations. Our findings support a role for the physical environment in shaping the affective meaning of colour.Peer reviewe

The Body Action and Posture Coding System (BAP): Development and Reliability

Several methods are available for coding body movement in nonverbal behavior research, but there is no consensus on a reliable coding system that can be used for the study of emotion expression. Adopting an integrative approach, we developed a new method, the Body Action and Posture (BAP) coding system, for the time-aligned micro description of body movement on an anatomical level (different articulations of body parts), a form level (direction and orientation of movement), and a functional level (communicative and self-regulatory functions). We applied the system to a new corpus of acted emotion portrayals, examined its comprehensiveness and demonstrated intercoder reliability at three levels: a) occurrence, b) temporal precision and c) segmentation. We discuss issues for further validation and propose some research applications

Expression and perception of emotion from body movement : categorical and dynamic correlates of emotion

For this thesis research we investigate the role of body movement in emotion communication. The Brunswikian Lens Model serves as a framework to distinguish both processes of emotion expression and emotion perception. The studies of this thesis are organized along tree main research lines. First, we develop descriptive tools for accurate measurement of body movement. In one study we describe the development and validation of a multilevel coding system for the categorical description of body posture and action with respect to the anatomical articulation, the movement form, and the movement function. Two other studies take a dynamic approach to measure spatiotemporal movement quality based on spatial annotation or semi-automatic movement tracking. Second, we use these tools to establish a representative inventory of the behavioral repertoire that can be employed when expressing emotions. In a first study, we investigate the possibility to differentiate emotions based on predictions of three major emotion theories (basic emotion theory, bi-dimensional theory, and componential appraisal theory). In a second study, we test the effect of expressed emotion on differences in gesture dynamics, related to the emotion dimensions of arousal, potency, and valence. Third, we test how systematic changes in expression associated to emotion drive the perception process. In two judgment studies we examine the impact of expressed emotion on the subjective representation of gesture dynamics, and on the attribution of emotion. In conclusion, this thesis identifies several categorical and dynamic correlates of emotion in body movement, and shows the utility of adopting a componential approach to the study of bodily emotion expression and perception