Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke

[EN] Social cognition is the innate human ability to interpret the emotional state of others from contextual verbal and non-verbal information, and to self-regulate accordingly. Facial expressions are one of the most relevant sources of non-verbal communication, and their interpretation has been extensively investigated in the literature, using both behavioral and physiological measures, such as those derived from visual activity and visual responses. The decoding of facial expressions of emotion is performed by conscious and unconscious cognitive processes that involve a complex brain network that can be damaged after cerebrovascular accidents. A diminished ability to identify facial expressions of emotion has been reported after stroke, which has traditionally been attributed to impaired emotional processing. While this can be true, an alteration in visual behavior after brain injury could also negatively contribute to this ability. This study investigated the accuracy, distribution of responses, visual behavior, and pupil dilation of individuals with stroke while identifying emotional facial expressions. Our results corroborated impaired performance after stroke and exhibited decreased attention to the eyes, evidenced by a diminished time and number of fixations made in this area in comparison to healthy subjects and comparable pupil dilation. The differences in visual behavior reached statistical significance in some emotions when comparing individuals with stroke with impaired performance with healthy subjects, but not when individuals post-stroke with comparable performance were considered. The performance dependence of visual behavior, although not determinant, might indicate that altered visual behavior could be a negatively contributing factor for emotion recognition from facial expressions.This study was funded by Conselleria de Educacion, Cultura y Deporte of Generalitat Valenciana of Spain (Project SEJI/2019/017), and Universitat Politecnica de Valencia (Grant PAID-10-18).Maza, A.; Moliner, B.; Ferri, J.; Llorens Rodríguez, R. (2020). Visual Behavior, Pupil Dilation, and Ability to Identify Emotions From Facial Expressions After Stroke. Frontiers in Neurology. 10:1-12. https://doi.org/10.3389/fneur.2019.01415S11210Nijsse, B., Spikman, J. M., Visser-Meily, J. M. A., de Kort, P. L. M., & van Heugten, C. M. (2019). Social cognition impairments are associated with behavioural changes in the long term after stroke. PLOS ONE, 14(3), e0213725. doi:10.1371/journal.pone.0213725Feldman, R. S., White, J. B., & Lobato, D. (1982). Social Skills and Nonverbal Behavior. Development of Nonverbal Behavior in Children, 259-277. doi:10.1007/978-1-4757-1761-7_10Vuilleumier, P., & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: Evidence from functional neuroimaging. Neuropsychologia, 45(1), 174-194. doi:10.1016/j.neuropsychologia.2006.06.003Frith, C. D., & Frith, U. (2006). How we predict what other people are going to do. Brain Research, 1079(1), 36-46. doi:10.1016/j.brainres.2005.12.126Zinchenko, O., Yaple, Z. A., & Arsalidou, M. (2018). Brain Responses to Dynamic Facial Expressions: A Normative Meta-Analysis. Frontiers in Human Neuroscience, 12. doi:10.3389/fnhum.2018.00227Ekman, P. (1993). Facial expression and emotion. American Psychologist, 48(4), 384-392. doi:10.1037/0003-066x.48.4.384Srinivasan, R., & Martinez, A. M. (2021). Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild. IEEE Transactions on Affective Computing, 12(3), 707-721. doi:10.1109/taffc.2018.2887267Smith, M. L., Grühn, D., Bevitt, A., Ellis, M., Ciripan, O., Scrimgeour, S., … Ewing, L. (2018). Transmitting and decoding facial expressions of emotion during healthy aging: More similarities than differences. Journal of Vision, 18(9), 10. doi:10.1167/18.9.10Thompson, A. E., & Voyer, D. (2014). Sex differences in the ability to recognise non-verbal displays of emotion: A meta-analysis. Cognition and Emotion, 28(7), 1164-1195. doi:10.1080/02699931.2013.875889Doležal, J., & Fabian, V. (2015). 41. Application of eye tracking in neuroscience. Clinical Neurophysiology, 126(3), e44. doi:10.1016/j.clinph.2014.10.200Guo, K. (2012). Holistic Gaze Strategy to Categorize Facial Expression of Varying Intensities. PLoS ONE, 7(8), e42585. doi:10.1371/journal.pone.0042585Guo, K., Soornack, Y., & Settle, R. (2019). Expression-dependent susceptibility to face distortions in processing of facial expressions of emotion. Vision Research, 157, 112-122. doi:10.1016/j.visres.2018.02.001Eisenbarth, H., & Alpers, G. W. (2011). Happy mouth and sad eyes: Scanning emotional facial expressions. Emotion, 11(4), 860-865. doi:10.1037/a0022758Schurgin, M. W., Nelson, J., Iida, S., Ohira, H., Chiao, J. Y., & Franconeri, S. L. (2014). Eye movements during emotion recognition in faces. Journal of Vision, 14(13), 14-14. doi:10.1167/14.13.14Guo, K., & Shaw, H. (2015). Face in profile view reduces perceived facial expression intensity: An eye-tracking study. Acta Psychologica, 155, 19-28. doi:10.1016/j.actpsy.2014.12.001Guo, K. (2013). Size-Invariant Facial Expression Categorization and Associated Gaze Allocation within Social Interaction Space. Perception, 42(10), 1027-1042. doi:10.1068/p7552Sirois, S., & Brisson, J. (2014). Pupillometry. WIREs Cognitive Science, 5(6), 679-692. doi:10.1002/wcs.1323Eckstein, M. K., Guerra-Carrillo, B., Miller Singley, A. T., & Bunge, S. A. (2017). Beyond eye gaze: What else can eyetracking reveal about cognition and cognitive development? Developmental Cognitive Neuroscience, 25, 69-91. doi:10.1016/j.dcn.2016.11.001Ariel, R., & Castel, A. D. (2013). Eyes wide open: enhanced pupil dilation when selectively studying important information. Experimental Brain Research, 232(1), 337-344. doi:10.1007/s00221-013-3744-5Zekveld, A. A., & Kramer, S. E. (2014). Cognitive processing load across a wide range of listening conditions: Insights from pupillometry. Psychophysiology, 51(3), 277-284. doi:10.1111/psyp.12151De Gee, J. W., Knapen, T., & Donner, T. H. (2014). Decision-related pupil dilation reflects upcoming choice and individual bias. Proceedings of the National Academy of Sciences, 111(5), E618-E625. doi:10.1073/pnas.1317557111Bradley, M. M., Miccoli, L., Escrig, M. A., & Lang, P. J. (2008). The pupil as a measure of emotional arousal and autonomic activation. Psychophysiology, 45(4), 602-607. doi:10.1111/j.1469-8986.2008.00654.xDuque, A., Sanchez, A., & Vazquez, C. (2014). Gaze-fixation and pupil dilation in the processing of emotional faces: The role of rumination. Cognition and Emotion, 28(8), 1347-1366. doi:10.1080/02699931.2014.881327Lanata, A., Armato, A., Valenza, G., & Scilingo, E. P. (2011). Eye tracking and pupil size variation as response to affective stimuli: a preliminary study. Proceedings of the 5th International ICST Conference on Pervasive Computing Technologies for Healthcare. doi:10.4108/icst.pervasivehealth.2011.246056Peinkhofer, C., Knudsen, G. M., Moretti, R., & Kondziella, D. (2019). Cortical modulation of pupillary function: systematic review. PeerJ, 7, e6882. doi:10.7717/peerj.6882Grill-Spector, K., Knouf, N., & Kanwisher, N. (2004). The fusiform face area subserves face perception, not generic within-category identification. Nature Neuroscience, 7(5), 555-562. doi:10.1038/nn1224Ferretti, V., & Papaleo, F. (2018). Understanding others: emotion recognition abilities in humans and other animals. Genes, Brain and Behavior, e12544. doi:10.1111/gbb.12544Sergerie, K., Chochol, C., & Armony, J. L. (2008). The role of the amygdala in emotional processing: A quantitative meta-analysis of functional neuroimaging studies. Neuroscience & Biobehavioral Reviews, 32(4), 811-830. doi:10.1016/j.neubiorev.2007.12.002Rapcsak, S. Z., Galper, S. R., Comer, J. F., Reminger, S. L., Nielsen, L., Kaszniak, A. W., … Cohen, R. A. (2000). Fear recognition deficits after focal brain damage: A cautionary note. Neurology, 54(3), 575-575. doi:10.1212/wnl.54.3.575Radice-Neumann, D., Zupan, B., Tomita, M., & Willer, B. (2009). Training Emotional Processing in Persons With Brain Injury. Journal of Head Trauma Rehabilitation, 24(5), 313-323. doi:10.1097/htr.0b013e3181b09160Yuvaraj, R., Murugappan, M., Norlinah, M. I., Sundaraj, K., & Khairiyah, M. (2013). Review of Emotion Recognition in Stroke Patients. Dementia and Geriatric Cognitive Disorders, 36(3-4), 179-196. doi:10.1159/000353440Babbage, D. R., Yim, J., Zupan, B., Neumann, D., Tomita, M. R., & Willer, B. (2011). Meta-analysis of facial affect recognition difficulties after traumatic brain injury. Neuropsychology, 25(3), 277-285. doi:10.1037/a0021908Milders, M., Fuchs, S., & Crawford, J. R. (2003). Neuropsychological Impairments and Changes in Emotional and Social Behaviour Following Severe Traumatic Brain Injury. Journal of Clinical and Experimental Neuropsychology, 25(2), 157-172. doi:10.1076/jcen.25.2.157.13642Genova, H. M., Genualdi, A., Goverover, Y., Chiaravalloti, N. D., Marino, C., & Lengenfelder, J. (2016). An investigation of the impact of facial affect recognition impairments in moderate to severe TBI on fatigue, depression, and quality of life. Social Neuroscience, 12(3), 303-307. doi:10.1080/17470919.2016.1173584Rigon, A., Voss, M. W., Turkstra, L. S., Mutlu, B., & Duff, M. C. (2018). Different aspects of facial affect recognition impairment following traumatic brain injury: The role of perceptual and interpretative abilities. Journal of Clinical and Experimental Neuropsychology, 40(8), 805-819. doi:10.1080/13803395.2018.1437120Rosenberg, H., McDonald, S., Dethier, M., Kessels, R. P. C., & Westbrook, R. F. (2014). Facial Emotion Recognition Deficits following Moderate–Severe Traumatic Brain Injury (TBI): Re-examining the Valence Effect and the Role of Emotion Intensity. Journal of the International Neuropsychological Society, 20(10), 994-1003. doi:10.1017/s1355617714000940Lancelot, C., & Gilles, C. (2018). How does visual context influence recognition of facial emotion in people with traumatic brain injury? Brain Injury, 33(1), 4-11. doi:10.1080/02699052.2018.1531308McDonald, S. (2013). Impairments in Social Cognition Following Severe Traumatic Brain Injury. Journal of the International Neuropsychological Society, 19(3), 231-246. doi:10.1017/s1355617712001506Vallat-Azouvi, C., Azouvi, P., Le-Bornec, G., & Brunet-Gouet, E. (2018). Treatment of social cognition impairments in patients with traumatic brain injury: a critical review. Brain Injury, 33(1), 87-93. doi:10.1080/02699052.2018.1531309Godin, B., Oishi, K., Oishi, K., Davis, C., Gomez, Y., Trupe, L., … Tippett, D. (2018). Impaired Recognition of Emotional Faces after Stroke Involving Right Amygdala or Insula. Seminars in Speech and Language, 39(01), 087-100. doi:10.1055/s-0037-1608859Abbott, J. D., Cumming, G., Fidler, F., & Lindell, A. K. (2013). The perception of positive and negative facial expressions in unilateral brain-damaged patients: A meta-analysis. Laterality: Asymmetries of Body, Brain and Cognition, 18(4), 437-459. doi:10.1080/1357650x.2012.703206Abbott, J. D., Wijeratne, T., Hughes, A., Perre, D., & Lindell, A. K. (2014). The perception of positive and negative facial expressions by unilateral stroke patients. Brain and Cognition, 86, 42-54. doi:10.1016/j.bandc.2014.01.017Delazer, M., Sojer, M., Ellmerer, P., Boehme, C., & Benke, T. (2018). Eye-Tracking Provides a Sensitive Measure of Exploration Deficits After Acute Right MCA Stroke. Frontiers in Neurology, 9. doi:10.3389/fneur.2018.00359Lech, M., Kucewicz, M. T., & Czyżewski, A. (2019). Human Computer Interface for Tracking Eye Movements Improves Assessment and Diagnosis of Patients With Acquired Brain Injuries. Frontiers in Neurology, 10. doi:10.3389/fneur.2019.00006Spikman, J. M., Milders, M. V., Visser-Keizer, A. C., Westerhof-Evers, H. J., Herben-Dekker, M., & van der Naalt, J. (2013). Deficits in Facial Emotion Recognition Indicate Behavioral Changes and Impaired Self-Awareness after Moderate to Severe Traumatic Brain Injury. PLoS ONE, 8(6), e65581. doi:10.1371/journal.pone.0065581Knox, L., & Douglas, J. (2009). Long-term ability to interpret facial expression after traumatic brain injury and its relation to social integration. Brain and Cognition, 69(2), 442-449. doi:10.1016/j.bandc.2008.09.009Struchen, M. A., Clark, A. N., Sander, A. M., Mills, M. R., Evans, G., & Kurtz, D. (2008). Relation of executive functioning and social communication measures to functional outcomes following traumatic brain injury. NeuroRehabilitation, 23(2), 185-198. doi:10.3233/nre-2008-23208Ferro, J. M., Caeiro, L., & Santos, C. (2009). Poststroke Emotional and Behavior Impairment: A Narrative Review. Cerebrovascular Diseases, 27(1), 197-203. doi:10.1159/000200460Bortolon, C., Capdevielle, D., & Raffard, S. (2015). Face recognition in schizophrenia disorder: A comprehensive review of behavioral, neuroimaging and neurophysiological studies. Neuroscience & Biobehavioral Reviews, 53, 79-107. doi:10.1016/j.neubiorev.2015.03.006Harms, M. B., Martin, A., & Wallace, G. L. (2010). Facial Emotion Recognition in Autism Spectrum Disorders: A Review of Behavioral and Neuroimaging Studies. Neuropsychology Review, 20(3), 290-322. doi:10.1007/s11065-010-9138-6Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). «Mini-mental state». Journal of Psychiatric Research, 12(3), 189-198. doi:10.1016/0022-3956(75)90026-6Romero, M., Sánchez, A., Marín, C., Navarro, M. D., Ferri, J., & Noé, E. (2012). Clinical usefulness of the Spanish version of the Mississippi Aphasia Screening Test (MASTsp): validation in stroke patients. Neurología (English Edition), 27(4), 216-224. doi:10.1016/j.nrleng.2011.06.001Aguillon-Hernandez, N., Roché, L., Bonnet-Brilhault, F., Roux, S., Barthelemy, C., & Martineau, J. (2016). Eye Movement Monitoring and Maturation of Human Face Exploration. Medical Principles and Practice, 25(6), 548-554. doi:10.1159/000447971Mathôt, S., Fabius, J., Van Heusden, E., & Van der Stigchel, S. (2018). Safe and sensible preprocessing and baseline correction of pupil-size data. Behavior Research Methods, 50(1), 94-106. doi:10.3758/s13428-017-1007-2Green, C., & Guo, K. (2016). Factors contributing to individual differences in facial expression categorisation. Cognition and Emotion, 32(1), 37-48. doi:10.1080/02699931.2016.1273200Burley, D. T., Gray, N. S., & Snowden, R. J. (2017). As Far as the Eye Can See: Relationship between Psychopathic Traits and Pupil Response to Affective Stimuli. PLOS ONE, 12(1), e0167436. doi:10.1371/journal.pone.0167436Partala, T., & Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies, 59(1-2), 185-198. doi:10.1016/s1071-5819(03)00017-xGotham, K. O., Siegle, G. J., Han, G. T., Tomarken, A. J., Crist, R. N., Simon, D. M., & Bodfish, J. W. (2018). Pupil response to social-emotional material is associated with rumination and depressive symptoms in adults with autism spectrum disorder. PLOS ONE, 13(8), e0200340. doi:10.1371/journal.pone.0200340Demenescu, L. R., Mathiak, K. A., & Mathiak, K. (2014). Age- and Gender-Related Variations of Emotion Recognition in Pseudowords and Faces. Experimental Aging Research, 40(2), 187-207. doi:10.1080/0361073x.2014.882210Grainger, S. A., Henry, J. D., Phillips, L. H., Vanman, E. J., & Allen, R. (2015). Age Deficits in Facial Affect Recognition: The Influence of Dynamic Cues. The Journals of Gerontology Series B: Psychological Sciences and Social Sciences, gbv100. doi:10.1093/geronb/gbv100Wagner, H. L. (1993). On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior, 17(1), 3-28. doi:10.1007/bf00987006Bradley, M. M., & Lang, P. J. (2015). Memory, emotion, and pupil diameter: Repetition of natural scenes. Psychophysiology, 52(9), 1186-1193. doi:10.1111/psyp.12442Larsen, R. S., & Waters, J. (2018). Neuromodulatory Correlates of Pupil Dilation. Frontiers in Neural Circuits, 12. doi:10.3389/fncir.2018.0002

To Fear is to Gain? The Role of Fear Recognition in Risky Decision Making in TBI Patients and Healthy Controls

Fear is an important emotional reaction that guides decision making in situations of ambiguity or uncertainty. Both recognition of facial expressions of fear and decision making ability can be impaired after traumatic brain injury (TBI), in particular when the frontal lobe is damaged. So far, it has not been investigated how recognition of fear influences risk behavior in healthy subjects and TBI patients. The ability to recognize fear is thought to be related to the ability to experience fear and to use it as a warning signal to guide decision making. We hypothesized that a better ability to recognize fear would be related to a better regulation of risk behavior, with healthy controls outperforming TBI patients. To investigate this, 59 healthy subjects and 49 TBI patients were assessed with a test for emotion recognition (Facial Expression of Emotion: Stimuli and Tests) and a gambling task (Iowa Gambling Task (IGT)). The results showed that, regardless of post traumatic amnesia duration or the presence of frontal lesions, patients were more impaired than healthy controls on both fear recognition and decision making. In both groups, a significant relationship was found between better fear recognition, the development of an advantageous strategy across the IGT and less risk behavior in the last blocks of the IGT. Educational level moderated this relationship in the final block of the IGT. This study has important clinical implications, indicating that impaired decision making and risk behavior after TBI can be preceded by deficits in the processing of fear

A sensorimotor control framework for understanding emotional communication and regulation

JHGW and CFH are supported by the Northwood Trust. TEVR was supported by a National Health and Medical Research Council (NHMRC) Early Career Fellowship (1088785). RP and MW were supported by the the Australian Research Council (ARC) Centre of Excellence for Cognition and its Disorders (CE110001021)Peer reviewedPublisher PD

Training emotional processing in persons with brain injury

Zupan, BA ORCiD: 0000-0002-4603-333XAIMS: To determine the effectiveness of 2 interventions for different aspects of emotion-processing deficits in adults with acquired brain injury (ABI). PARTICIPANTS: Nineteen participants with ABI (minimum 1 year postinjury) from Western New York and Southern Ontario, Canada. INTERVENTIONS: (1) Emotion processing from faces ("facial affect recognition" or FAR) and (2) emotion processing from written context by using "stories of emotional inference" (SEI). Ten randomly assigned participants received the FAR intervention, and 9 received the SEI protocol. Both interventions were administered 1 hour per day, 3 times per week, and completed in 6 to 9 sessions, and both incorporated participants personal emotional experiences into training. OUTCOME MEASURES: (1) Facial affect, (2) vocal affect, (3) affect from videos, (4) emotional inference from context, and (5) emotional behavior. There were 2 pretests, a posttest, and a 2-week follow-up. RESULTS: FAR participants showed significantly improved emotion recognition from faces, ability to infer emotions from context, and socioemotional behavior, while the SEI group members exhibited significantly improved ability to infer how they would feel in a given context. CONCLUSION: Training can improve emotion perception in persons with ABI. Although further research is needed, the interventions are clinically practical and show promise for the population with ABI. Copyright © 2009 Wolters Kluwer Health | Lippincott Williams & Wilkins

Training emotional processing in persons with brain injury

AIMS: To determine the effectiveness of 2 interventions for different aspects of emotion-processing deficits in adults with acquired brain injury (ABI). PARTICIPANTS: Nineteen participants with ABI (minimum 1 year postinjury) from Western New York and Southern Ontario, Canada. INTERVENTIONS: (1) Emotion processing from faces ("facial affect recognition" or FAR) and (2) emotion processing from written context by using "stories of emotional inference" (SEI). Ten randomly assigned participants received the FAR intervention, and 9 received the SEI protocol. Both interventions were administered 1 hour per day, 3 times per week, and completed in 6 to 9 sessions, and both incorporated participants personal emotional experiences into training. OUTCOME MEASURES: (1) Facial affect, (2) vocal affect, (3) affect from videos, (4) emotional inference from context, and (5) emotional behavior. There were 2 pretests, a posttest, and a 2-week follow-up. RESULTS: FAR participants showed significantly improved emotion recognition from faces, ability to infer emotions from context, and socioemotional behavior, while the SEI group members exhibited significantly improved ability to infer how they would feel in a given context. CONCLUSION: Training can improve emotion perception in persons with ABI. Although further research is needed, the interventions are clinically practical and show promise for the population with ABI. Copyright © 2009 Wolters Kluwer Health | Lippincott Williams & Wilkins

Overview of impaired facial affect recognition in persons with traumatic brain injury

Primary objective: To review the literature of affect recognition for persons with traumatic brain injury (TBI). It is suggested that impairment of affect recognition could be a significant problem for the TBI population and treatment strategies are recommended based on research for persons with autism. Main outcomes and results: Research demonstrates that persons with TBI often have difficulty determining emotion from facial expressions. Studies show that poor i nterpersonal skills, which are associated with impaired affect recognition, are linked to a variety of negative outcomes. Theories suggest that facial affect recognition is achieved by interpreting important facial features and processing one's own emotions. These skills are often affected by TBI, depending on the areas damaged. Affect recognition impairments have also been identified in persons with autism. Successful interventions have already been developed for the autism population. Comparable neuroanatomical and behavioural findings between TBI and autism suggest that treatment approaches for autism may also benefit those with TBI. Conclusions: Impaired facial affect recognition appears to be a significant problem for persons with TBI. Theories of affect recognition, strategies used in autism and teaching techniques commonly used in TBI need to be considered when developing treatments to improve affect recognition in persons with brain injury