Seeing through the shades of situated affectivity. Sunglasses as a socio-affective artifact

Debates on situated affectivity have mainly focused on tools that exert some positive influence on affective experience. Far less attention has been paid to artifacts that interact with the expression of affect, or to those that exert some negative influence. To shed light on that shadowy corner of our affective social lives, I describe the workings of an atypical socio-affective artifact, namely, sunglasses. Drawing on insights from psychology and other social sciences, I construe sunglasses as a social shield that helps us block spontaneous emotional expressions, as well as affecting other social processes that heavily depend on the eye region: gaze direction detection, identity recognition, and the sense of intimacy afforded by eye contact

Investigating emotion perception via the two-dimensional affect and feeling space: an example of a cross-cultural study among Chinese and Non-Chinese participants

The categorical approach to cross-cultural emotion perception research has mainly relied on constrained experimental tasks, which have arguably biased previous findings and attenuated cross-cultural differences. On the other hand, in the constructionist approach, conclusions on the universal nature of valence and arousal have mainly been indirectly drawn based on participants' word-matching or free-sorting behaviors, but studies based on participants' continuous valence and arousal ratings are very scarce. When it comes to self-reports of specific emotion perception, constructionists tend to rely on free labeling, which has its own limitations. In an attempt to move beyond the limitations of previous methods, a new instrument called the Two-Dimensional Affect and Feeling Space (2DAFS) has been developed. The 2DAFS is a useful, innovative, and user-friendly instrument that can easily be integrated in online surveys and allows for the collection of both continuous valence and arousal ratings and categorical emotion perception data in a quick and flexible way. In order to illustrate the usefulness of this tool, a cross-cultural emotion perception study based on the 2DAFS is reported. The results indicate the cross-cultural variation in valence and arousal perception, suggesting that the minimal universality hypothesis might need to be more nuanced

Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements

It is commonly assumed that a person’s emotional state can be readily inferred from his or her facial movements, typically called emotional expressions or facial expressions. This assumption influences legal judgments, policy decisions, national security protocols, and educational practices; guides the diagnosis and treatment of psychiatric illness, as well as the development of commercial applications; and pervades everyday social interactions as well as research in other scientific fields such as artificial intelligence, neuroscience, and computer vision. In this article, we survey examples of this widespread assumption, which we refer to as the common view, and we then examine the scientific evidence that tests this view, focusing on the six most popular emotion categories used by consumers of emotion research: anger, disgust, fear, happiness, sadness, and surprise. The available scientific evidence suggests that people do sometimes smile when happy, frown when sad, scowl when angry, and so on, as proposed by the common view, more than what would be expected by chance. Yet how people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation. Furthermore, similar configurations of facial movements variably express instances of more than one emotion category. In fact, a given configuration of facial movements, such as a scowl, often communicates something other than an emotional state. Scientists agree that facial movements convey a range of information and are important for social communication, emotional or otherwise. But our review suggests an urgent need for research that examines how people actually move their faces to express emotions and other social information in the variety of contexts that make up everyday life, as well as careful study of the mechanisms by which people perceive instances of emotion in one another. We make specific research recommendations that will yield a more valid picture of how people move their faces to express emotions and how they infer emotional meaning from facial movements in situations of everyday life. This research is crucial to provide consumers of emotion research with the translational information they require

The role of facial movements in emotion recognition

Most past research on emotion recognition has used photographs of posed expressions intended to depict the apex of the emotional display. Although these studies have provided important insights into how emotions are perceived in the face, they necessarily leave out any role of dynamic information. In this Review, we synthesize evidence from vision science, affective science and neuroscience to ask when, how and why dynamic information contributes to emotion recognition, beyond the information conveyed in static images. Dynamic displays offer distinctive temporal information such as the direction, quality and speed of movement, which recruit higher-level cognitive processes and support social and emotional inferences that enhance judgements of facial affect. The positive influence of dynamic information on emotion recognition is most evident in suboptimal conditions when observers are impaired and/or facial expressions are degraded or subtle. Dynamic displays further recruit early attentional and motivational resources in the perceiver, facilitating the prompt detection and prediction of others’ emotional states, with benefits for social interaction. Finally, because emotions can be expressed in various modalities, we examine the multimodal integration of dynamic and static cues across different channels, and conclude with suggestions for future research

Emotional expressions reconsidered: challenges to inferring emotion from human facial movements

It is commonly assumed that a person’s emotional state can be readily inferred from his or her facial movements, typically called emotional expressions or facial expressions. This assumption influences legal judgments, policy decisions, national security protocols, and educational practices; guides the diagnosis and treatment of psychiatric illness, as well as the development of commercial applications; and pervades everyday social interactions as well as research in other scientific fields such as artificial intelligence, neuroscience, and computer vision. In this article, we survey examples of this widespread assumption, which we refer to as the common view, and we then examine the scientific evidence that tests this view, focusing on the six most popular emotion categories used by consumers of emotion research: anger, disgust, fear, happiness, sadness, and surprise. The available scientific evidence suggests that people do sometimes smile when happy, frown when sad, scowl when angry, and so on, as proposed by the common view, more than what would be expected by chance. Yet how people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation. Furthermore, similar configurations of facial movements variably express instances of more than one emotion category. In fact, a given configuration of facial movements, such as a scowl, often communicates something other than an emotional state. Scientists agree that facial movements convey a range of information and are important for social communication, emotional or otherwise. But our review suggests an urgent need for research that examines how people actually move their faces to express emotions and other social information in the variety of contexts that make up everyday life, as well as careful study of the mechanisms by which people perceive instances of emotion in one another. We make specific research recommendations that will yield a more valid picture of how people move their faces to express emotions and how they infer emotional meaning from facial movements in situations of everyday life. This research is crucial to provide consumers of emotion research with the translational information they require

Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke

[EN] Social cognition is the innate human ability to interpret the emotional state of others from contextual verbal and non-verbal information, and to self-regulate accordingly. Facial expressions are one of the most relevant sources of non-verbal communication, and their interpretation has been extensively investigated in the literature, using both behavioral and physiological measures, such as those derived from visual activity and visual responses. The decoding of facial expressions of emotion is performed by conscious and unconscious cognitive processes that involve a complex brain network that can be damaged after cerebrovascular accidents. A diminished ability to identify facial expressions of emotion has been reported after stroke, which has traditionally been attributed to impaired emotional processing. While this can be true, an alteration in visual behavior after brain injury could also negatively contribute to this ability. This study investigated the accuracy, distribution of responses, visual behavior, and pupil dilation of individuals with stroke while identifying emotional facial expressions. Our results corroborated impaired performance after stroke and exhibited decreased attention to the eyes, evidenced by a diminished time and number of fixations made in this area in comparison to healthy subjects and comparable pupil dilation. The differences in visual behavior reached statistical significance in some emotions when comparing individuals with stroke with impaired performance with healthy subjects, but not when individuals post-stroke with comparable performance were considered. The performance dependence of visual behavior, although not determinant, might indicate that altered visual behavior could be a negatively contributing factor for emotion recognition from facial expressions.This study was funded by Conselleria de Educacion, Cultura y Deporte of Generalitat Valenciana of Spain (Project SEJI/2019/017), and Universitat Politecnica de Valencia (Grant PAID-10-18).Maza, A.; Moliner, B.; Ferri, J.; Llorens Rodríguez, R. (2020). Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke. Frontiers in Neurology. 10:1-12. https://doi.org/10.3389/fneur.2019.01415S11210Nijsse, B., Spikman, J. M., Visser-Meily, J. M. A., de Kort, P. L. M., & van Heugten, C. M. (2019). Social cognition impairments are associated with behavioural changes in the long term after stroke. PLOS ONE, 14(3), e0213725. doi:10.1371/journal.pone.0213725Feldman, R. S., White, J. B., & Lobato, D. (1982). Social Skills and Nonverbal Behavior. Development of Nonverbal Behavior in Children, 259-277. doi:10.1007/978-1-4757-1761-7_10Vuilleumier, P., & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia, 45(1), 174-194. doi:10.1016/j.neuropsychologia.2006.06.003Frith, C. D., & Frith, U. (2006). How we predict what other people are going to do. Brain Research, 1079(1), 36-46. doi:10.1016/j.brainres.2005.12.126Zinchenko, O., Yaple, Z. A., & Arsalidou, M. (2018). Brain Responses to Dynamic Facial Expressions: A Normative Meta-Analysis. Frontiers in Human Neuroscience, 12. doi:10.3389/fnhum.2018.00227Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48(4), 384-392. doi:10.1037/0003-066x.48.4.384Srinivasan, R., & Martinez, A. M. (2021). Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild. IEEE Transactions on Affective Computing, 12(3), 707-721. doi:10.1109/taffc.2018.2887267Smith, M. L., Grühn, D., Bevitt, A., Ellis, M., Ciripan, O., Scrimgeour, S., … Ewing, L. (2018). Transmitting and decoding facial expressions of emotion during healthy aging: More similarities than differences. Journal of Vision, 18(9), 10. doi:10.1167/18.9.10Thompson, A. E., & Voyer, D. (2014). Sex differences in the ability to recognise non-verbal displays of emotion: A meta-analysis. Cognition and Emotion, 28(7), 1164-1195. doi:10.1080/02699931.2013.875889Doležal, J., & Fabian, V. (2015). 41. Application of eye tracking in neuroscience. Clinical Neurophysiology, 126(3), e44. doi:10.1016/j.clinph.2014.10.200Guo, K. (2012). Holistic Gaze Strategy to Categorize Facial Expression of Varying Intensities. PLoS ONE, 7(8), e42585. doi:10.1371/journal.pone.0042585Guo, K., Soornack, Y., & Settle, R. (2019). Expression-dependent susceptibility to face distortions in processing of facial expressions of emotion. Vision Research, 157, 112-122. doi:10.1016/j.visres.2018.02.001Eisenbarth, H., & Alpers, G. W. (2011). Happy mouth and sad eyes: Scanning emotional facial expressions. Emotion, 11(4), 860-865. doi:10.1037/a0022758Schurgin, M. W., Nelson, J., Iida, S., Ohira, H., Chiao, J. Y., & Franconeri, S. L. (2014). Eye movements during emotion recognition in faces. Journal of Vision, 14(13), 14-14. doi:10.1167/14.13.14Guo, K., & Shaw, H. (2015). Face in profile view reduces perceived facial expression intensity: An eye-tracking study. Acta Psychologica, 155, 19-28. doi:10.1016/j.actpsy.2014.12.001Guo, K. (2013). Size-Invariant Facial Expression Categorization and Associated Gaze Allocation within Social Interaction Space. Perception, 42(10), 1027-1042. doi:10.1068/p7552Sirois, S., & Brisson, J. (2014). Pupillometry. WIREs Cognitive Science, 5(6), 679-692. doi:10.1002/wcs.1323Eckstein, M. K., Guerra-Carrillo, B., Miller Singley, A. T., & Bunge, S. A. (2017). Beyond eye gaze: What else can eyetracking reveal about cognition and cognitive development? Developmental Cognitive Neuroscience, 25, 69-91. doi:10.1016/j.dcn.2016.11.001Ariel, R., & Castel, A. D. (2013). Eyes wide open: enhanced pupil dilation when selectively studying important information. Experimental Brain Research, 232(1), 337-344. doi:10.1007/s00221-013-3744-5Zekveld, A. A., & Kramer, S. E. (2014). Cognitive processing load across a wide range of listening conditions: Insights from pupillometry. Psychophysiology, 51(3), 277-284. doi:10.1111/psyp.12151De Gee, J. W., Knapen, T., & Donner, T. H. (2014). Decision-related pupil dilation reflects upcoming choice and individual bias. Proceedings of the National Academy of Sciences, 111(5), E618-E625. doi:10.1073/pnas.1317557111Bradley, M. M., Miccoli, L., Escrig, M. A., & Lang, P. J. (2008). The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology, 45(4), 602-607. doi:10.1111/j.1469-8986.2008.00654.xDuque, A., Sanchez, A., & Vazquez, C. (2014). Gaze-fixation and pupil dilation in the processing of emotional faces: The role of rumination. Cognition and Emotion, 28(8), 1347-1366. doi:10.1080/02699931.2014.881327Lanata, A., Armato, A., Valenza, G., & Scilingo, E. P. (2011). Eye tracking and pupil size variation as response to affective stimuli: a preliminary study. Proceedings of the 5th International ICST Conference on Pervasive Computing Technologies for Healthcare. doi:10.4108/icst.pervasivehealth.2011.246056Peinkhofer, C., Knudsen, G. M., Moretti, R., & Kondziella, D. (2019). Cortical modulation of pupillary function: systematic review. PeerJ, 7, e6882. doi:10.7717/peerj.6882Grill-Spector, K., Knouf, N., & Kanwisher, N. (2004). The fusiform face area subserves face perception, not generic within-category identification. Nature Neuroscience, 7(5), 555-562. doi:10.1038/nn1224Ferretti, V., & Papaleo, F. (2018). Understanding others: emotion recognition abilities in humans and other animals. Genes, Brain and Behavior, e12544. doi:10.1111/gbb.12544Sergerie, K., Chochol, C., & Armony, J. L. (2008). The role of the amygdala in emotional processing: A quantitative meta-analysis of functional neuroimaging studies. Neuroscience & Biobehavioral Reviews, 32(4), 811-830. doi:10.1016/j.neubiorev.2007.12.002Rapcsak, S. Z., Galper, S. R., Comer, J. F., Reminger, S. L., Nielsen, L., Kaszniak, A. W., … Cohen, R. A. (2000). Fear recognition deficits after focal brain damage: A cautionary note. Neurology, 54(3), 575-575. doi:10.1212/wnl.54.3.575Radice-Neumann, D., Zupan, B., Tomita, M., & Willer, B. (2009). Training Emotional Processing in Persons With Brain Injury. Journal of Head Trauma Rehabilitation, 24(5), 313-323. doi:10.1097/htr.0b013e3181b09160Yuvaraj, R., Murugappan, M., Norlinah, M. I., Sundaraj, K., & Khairiyah, M. (2013). Review of Emotion Recognition in Stroke Patients. Dementia and Geriatric Cognitive Disorders, 36(3-4), 179-196. doi:10.1159/000353440Babbage, D. R., Yim, J., Zupan, B., Neumann, D., Tomita, M. R., & Willer, B. (2011). Meta-analysis of facial affect recognition difficulties after traumatic brain injury. Neuropsychology, 25(3), 277-285. doi:10.1037/a0021908Milders, M., Fuchs, S., & Crawford, J. R. (2003). Neuropsychological Impairments and Changes in Emotional and Social Behaviour Following Severe Traumatic Brain Injury. Journal of Clinical and Experimental Neuropsychology, 25(2), 157-172. doi:10.1076/jcen.25.2.157.13642Genova, H. M., Genualdi, A., Goverover, Y., Chiaravalloti, N. D., Marino, C., & Lengenfelder, J. (2016). An investigation of the impact of facial affect recognition impairments in moderate to severe TBI on fatigue, depression, and quality of life. Social Neuroscience, 12(3), 303-307. doi:10.1080/17470919.2016.1173584Rigon, A., Voss, M. W., Turkstra, L. S., Mutlu, B., & Duff, M. C. (2018). Different aspects of facial affect recognition impairment following traumatic brain injury: The role of perceptual and interpretative abilities. Journal of Clinical and Experimental Neuropsychology, 40(8), 805-819. doi:10.1080/13803395.2018.1437120Rosenberg, H., McDonald, S., Dethier, M., Kessels, R. P. C., & Westbrook, R. F. (2014). Facial Emotion Recognition Deficits following Moderate–Severe Traumatic Brain Injury (TBI): Re-examining the Valence Effect and the Role of Emotion Intensity. Journal of the International Neuropsychological Society, 20(10), 994-1003. doi:10.1017/s1355617714000940Lancelot, C., & Gilles, C. (2018). How does visual context influence recognition of facial emotion in people with traumatic brain injury? Brain Injury, 33(1), 4-11. doi:10.1080/02699052.2018.1531308McDonald, S. (2013). Impairments in Social Cognition Following Severe Traumatic Brain Injury. Journal of the International Neuropsychological Society, 19(3), 231-246. doi:10.1017/s1355617712001506Vallat-Azouvi, C., Azouvi, P., Le-Bornec, G., & Brunet-Gouet, E. (2018). Treatment of social cognition impairments in patients with traumatic brain injury: a critical review. Brain Injury, 33(1), 87-93. doi:10.1080/02699052.2018.1531309Godin, B., Oishi, K., Oishi, K., Davis, C., Gomez, Y., Trupe, L., … Tippett, D. (2018). Impaired Recognition of Emotional Faces after Stroke Involving Right Amygdala or Insula. Seminars in Speech and Language, 39(01), 087-100. doi:10.1055/s-0037-1608859Abbott, J. D., Cumming, G., Fidler, F., & Lindell, A. K. (2013). The perception of positive and negative facial expressions in unilateral brain-damaged patients: A meta-analysis. Laterality: Asymmetries of Body, Brain and Cognition, 18(4), 437-459. doi:10.1080/1357650x.2012.703206Abbott, J. D., Wijeratne, T., Hughes, A., Perre, D., & Lindell, A. K. (2014). The perception of positive and negative facial expressions by unilateral stroke patients. Brain and Cognition, 86, 42-54. doi:10.1016/j.bandc.2014.01.017Delazer, M., Sojer, M., Ellmerer, P., Boehme, C., & Benke, T. (2018). Eye-Tracking Provides a Sensitive Measure of Exploration Deficits After Acute Right MCA Stroke. Frontiers in Neurology, 9. doi:10.3389/fneur.2018.00359Lech, M., Kucewicz, M. T., & Czyżewski, A. (2019). Human Computer Interface for Tracking Eye Movements Improves Assessment and Diagnosis of Patients With Acquired Brain Injuries. Frontiers in Neurology, 10. doi:10.3389/fneur.2019.00006Spikman, J. M., Milders, M. V., Visser-Keizer, A. C., Westerhof-Evers, H. J., Herben-Dekker, M., & van der Naalt, J. (2013). Deficits in Facial Emotion Recognition Indicate Behavioral Changes and Impaired Self-Awareness after Moderate to Severe Traumatic Brain Injury. PLoS ONE, 8(6), e65581. doi:10.1371/journal.pone.0065581Knox, L., & Douglas, J. (2009). Long-term ability to interpret facial expression after traumatic brain injury and its relation to social integration. Brain and Cognition, 69(2), 442-449. doi:10.1016/j.bandc.2008.09.009Struchen, M. A., Clark, A. N., Sander, A. M., Mills, M. R., Evans, G., & Kurtz, D. (2008). Relation of executive functioning and social communication measures to functional outcomes following traumatic brain injury. NeuroRehabilitation, 23(2), 185-198. doi:10.3233/nre-2008-23208Ferro, J. M., Caeiro, L., & Santos, C. (2009). Poststroke Emotional and Behavior Impairment: A Narrative Review. Cerebrovascular Diseases, 27(1), 197-203. doi:10.1159/000200460Bortolon, C., Capdevielle, D., & Raffard, S. (2015). Face recognition in schizophrenia disorder: A comprehensive review of behavioral, neuroimaging and neurophysiological studies. Neuroscience & Biobehavioral Reviews, 53, 79-107. doi:10.1016/j.neubiorev.2015.03.006Harms, M. B., Martin, A., & Wallace, G. L. (2010). Facial Emotion Recognition in Autism Spectrum Disorders: A Review of Behavioral and Neuroimaging Studies. Neuropsychology Review, 20(3), 290-322. doi:10.1007/s11065-010-9138-6Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). «Mini-mental state». Journal of Psychiatric Research, 12(3), 189-198. doi:10.1016/0022-3956(75)90026-6Romero, M., Sánchez, A., Marín, C., Navarro, M. D., Ferri, J., & Noé, E. (2012). Clinical usefulness of the Spanish version of the Mississippi Aphasia Screening Test (MASTsp): validation in stroke patients. Neurología (English Edition), 27(4), 216-224. doi:10.1016/j.nrleng.2011.06.001Aguillon-Hernandez, N., Roché, L., Bonnet-Brilhault, F., Roux, S., Barthelemy, C., & Martineau, J. (2016). Eye Movement Monitoring and Maturation of Human Face Exploration. Medical Principles and Practice, 25(6), 548-554. doi:10.1159/000447971Mathôt, S., Fabius, J., Van Heusden, E., & Van der Stigchel, S. (2018). Safe and sensible preprocessing and baseline correction of pupil-size data. Behavior Research Methods, 50(1), 94-106. doi:10.3758/s13428-017-1007-2Green, C., & Guo, K. (2016). Factors contributing to individual differences in facial expression categorisation. Cognition and Emotion, 32(1), 37-48. doi:10.1080/02699931.2016.1273200Burley, D. T., Gray, N. S., & Snowden, R. J. (2017). As Far as the Eye Can See: Relationship between Psychopathic Traits and Pupil Response to Affective Stimuli. PLOS ONE, 12(1), e0167436. doi:10.1371/journal.pone.0167436Partala, T., & Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies, 59(1-2), 185-198. doi:10.1016/s1071-5819(03)00017-xGotham, K. O., Siegle, G. J., Han, G. T., Tomarken, A. J., Crist, R. N., Simon, D. M., & Bodfish, J. W. (2018). Pupil response to social-emotional material is associated with rumination and depressive symptoms in adults with autism spectrum disorder. PLOS ONE, 13(8), e0200340. doi:10.1371/journal.pone.0200340Demenescu, L. R., Mathiak, K. A., & Mathiak, K. (2014). Age- and Gender-Related Variations of Emotion Recognition in Pseudowords and Faces. Experimental Aging Research, 40(2), 187-207. doi:10.1080/0361073x.2014.882210Grainger, S. A., Henry, J. D., Phillips, L. H., Vanman, E. J., & Allen, R. (2015). Age Deficits in Facial Affect Recognition: The Influence of Dynamic Cues. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, gbv100. doi:10.1093/geronb/gbv100Wagner, H. L. (1993). On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior, 17(1), 3-28. doi:10.1007/bf00987006Bradley, M. M., & Lang, P. J. (2015). Memory, emotion, and pupil diameter: Repetition of natural scenes. Psychophysiology, 52(9), 1186-1193. doi:10.1111/psyp.12442Larsen, R. S., & Waters, J. (2018). Neuromodulatory Correlates of Pupil Dilation. Frontiers in Neural Circuits, 12. doi:10.3389/fncir.2018.0002