The Influence of Each Facial Feature on How We Perceive and Interpret Human Faces

[EN] Facial information is processed by our brain in such a way that we immediately make judgments about, for example, attractiveness or masculinity or interpret personality traits or moods of other people. The appearance of each facial feature has an effect on our perception of facial traits. This research addresses the problem of measuring the size of these effects for five facial features (eyes, eyebrows, nose, mouth, and jaw). Our proposal is a mixed feature-based and image-based approach that allows judgments to be made on complete real faces in the categorization tasks, more than on synthetic, noisy, or partial faces that can influence the assessment. Each facial feature of the faces is automatically classified considering their global appearance using principal component analysis. Using this procedure, we establish a reduced set of relevant specific attributes (each one describing a complete facial feature) to characterize faces. In this way, a more direct link can be established between perceived facial traits and what people intuitively consider an eye, an eyebrow, a nose, a mouth, or a jaw. A set of 92 male faces were classified using this procedure, and the results were related to their scores in 15 perceived facial traits. We show that the relevant features greatly depend on what we are trying to judge. Globally, the eyes have the greatest effect. However, other facial features are more relevant for some judgments like the mouth for happiness and femininity or the nose for dominance.This study was carried out using the Chicago Face Database developed at the University of Chicago by Debbie S. Ma, Joshua Correll, and Bernd Wittenbrink.Diego-Mas, JA.; Fuentes-Hurtado, FJ.; Naranjo Ornedo, V.; Alcañiz Raya, ML. (2020). The Influence of Each Facial Feature on How We Perceive and Interpret Human Faces. i-Perception. 11(5):1-18. https://doi.org/10.1177/2041669520961123S118115Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face Description with Local Binary Patterns: Application to Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2037-2041. doi:10.1109/tpami.2006.244Axelrod, V., & Yovel, G. (2010). External facial features modify the representation of internal facial features in the fusiform face area. NeuroImage, 52(2), 720-725. doi:10.1016/j.neuroimage.2010.04.027Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711-720. doi:10.1109/34.598228Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94(2), 115-147. doi:10.1037/0033-295x.94.2.115Blais, C., Roy, C., Fiset, D., Arguin, M., & Gosselin, F. (2012). The eyes are not the window to basic emotions. Neuropsychologia, 50(12), 2830-2838. doi:10.1016/j.neuropsychologia.2012.08.010Bovet, J., Barthes, J., Durand, V., Raymond, M., & Alvergne, A. (2012). Men’s Preference for Women’s Facial Features: Testing Homogamy and the Paternity Uncertainty Hypothesis. PLoS ONE, 7(11), e49791. doi:10.1371/journal.pone.0049791Brahnam, S., & Nanni, L. (2010). Predicting trait impressions of faces using local face recognition techniques. Expert Systems with Applications, 37(7), 5086-5093. doi:10.1016/j.eswa.2009.12.002Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77(3), 305-327. doi:10.1111/j.2044-8295.1986.tb02199.xCabeza, R., & Kato, T. (2000). Features are Also Important: Contributions of Featural and Configural Processing to Face Recognition. Psychological Science, 11(5), 429-433. doi:10.1111/1467-9280.00283Chihaoui, M., Elkefi, A., Bellil, W., & Ben Amar, C. (2016). A Survey of 2D Face Recognition Techniques. Computers, 5(4), 21. doi:10.3390/computers5040021Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681-685. doi:10.1109/34.927467Diamond, R., & Carey, S. (1986). Why faces are and are not special: An effect of expertise. Journal of Experimental Psychology: General, 115(2), 107-117. doi:10.1037/0096-3445.115.2.107Dixson, B. J. W., Sulikowski, D., Gouda‐Vossos, A., Rantala, M. J., & Brooks, R. C. (2016). The masculinity paradox: facial masculinity and beardedness interact to determine women’s ratings of men’s facial attractiveness. Journal of Evolutionary Biology, 29(11), 2311-2320. doi:10.1111/jeb.12958Dunn†, J. C. (1974). Well-Separated Clusters and Optimal Fuzzy Partitions. Journal of Cybernetics, 4(1), 95-104. doi:10.1080/01969727408546059Eberhardt, J. L., Davies, P. G., Purdie-Vaughns, V. J., & Johnson, S. L. (2006). Looking Deathworthy. Psychological Science, 17(5), 383-386. doi:10.1111/j.1467-9280.2006.01716.xFink, B., Neave, N., Manning, J. T., & Grammer, K. (2006). Facial symmetry and judgements of attractiveness, health and personality. Personality and Individual Differences, 41(3), 491-499. doi:10.1016/j.paid.2006.01.017Fox, E., & Damjanovic, L. (2006). The eyes are sufficient to produce a threat superiority effect. Emotion, 6(3), 534-539. doi:10.1037/1528-3542.6.3.534Fuentes-Hurtado, F., Diego-Mas, J. A., Naranjo, V., & Alcañiz, M. (2019). Automatic classification of human facial features based on their appearance. PLOS ONE, 14(1), e0211314. doi:10.1371/journal.pone.0211314Gill, D. (2017). Women and men integrate facial information differently in appraising the beauty of a face. Evolution and Human Behavior, 38(6), 756-760. doi:10.1016/j.evolhumbehav.2017.07.001Gosselin, F., & Schyns, P. G. (2001). Bubbles: a technique to reveal the use of information in recognition tasks. Vision Research, 41(17), 2261-2271. doi:10.1016/s0042-6989(01)00097-9Hagiwara, N., Kashy, D. A., & Cesario, J. (2012). The independent effects of skin tone and facial features on Whites’ affective reactions to Blacks. Journal of Experimental Social Psychology, 48(4), 892-898. doi:10.1016/j.jesp.2012.02.001Hayward, W. G., Rhodes, G., & Schwaninger, A. (2008). An own-race advantage for components as well as configurations in face recognition. Cognition, 106(2), 1017-1027. doi:10.1016/j.cognition.2007.04.002Jack, R. E., & Schyns, P. G. (2015). The Human Face as a Dynamic Tool for Social Communication. Current Biology, 25(14), R621-R634. doi:10.1016/j.cub.2015.05.052Jones, B. C., Little, A. C., Burt, D. M., & Perrett, D. I. (2004). When Facial Attractiveness is Only Skin Deep. Perception, 33(5), 569-576. doi:10.1068/p3463Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception. The Journal of Neuroscience, 17(11), 4302-4311. doi:10.1523/jneurosci.17-11-04302.1997Keating, C. F., & Doyle, J. (2002). The faces of desirable mates and dates contain mixed social status cues. Journal of Experimental Social Psychology, 38(4), 414-424. doi:10.1016/s0022-1031(02)00007-0Keil, M. S. (2009). «I Look in Your Eyes, Honey»: Internal Face Features Induce Spatial Frequency Preference for Human Face Processing. PLoS Computational Biology, 5(3), e1000329. doi:10.1371/journal.pcbi.1000329Kwart, D. G., Foulsham, T., & Kingstone, A. (2012). Age and Beauty are in the Eye of the Beholder. Perception, 41(8), 925-938. doi:10.1068/p7136Langlois, J. H., Kalakanis, L., Rubenstein, A. J., Larson, A., Hallam, M., & Smoot, M. (2000). Maxims or myths of beauty? A meta-analytic and theoretical review. Psychological Bulletin, 126(3), 390-423. doi:10.1037/0033-2909.126.3.390Levine, T. R., & Hullett, C. R. (2002). Eta Squared, Partial Eta Squared, and Misreporting of Effect Size in Communication Research. Human Communication Research, 28(4), 612-625. doi:10.1111/j.1468-2958.2002.tb00828.xLittle, A. C., Burriss, R. P., Jones, B. C., & Roberts, S. C. (2007). Facial appearance affects voting decisions. Evolution and Human Behavior, 28(1), 18-27. doi:10.1016/j.evolhumbehav.2006.09.002Lundqvist, D., Esteves, F., & Ohman, A. (1999). The Face of Wrath: Critical Features for Conveying Facial Threat. Cognition & Emotion, 13(6), 691-711. doi:10.1080/026999399379041Ma, D. S., Correll, J., & Wittenbrink, B. (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior Research Methods, 47(4), 1122-1135. doi:10.3758/s13428-014-0532-5Maloney, L. T., & Dal Martello, M. F. (2006). Kin recognition and the perceived facial similarity of children. Journal of Vision, 6(10), 4. doi:10.1167/6.10.4McKone, E., & Yovel, G. (2009). Why does picture-plane inversion sometimes dissociate perception of features and spacing in faces, and sometimes not? Toward a new theory of holistic processing. Psychonomic Bulletin & Review, 16(5), 778-797. doi:10.3758/pbr.16.5.778Meyers, E., & Wolf, L. (2007). Using Biologically Inspired Features for Face Processing. International Journal of Computer Vision, 76(1), 93-104. doi:10.1007/s11263-007-0058-8Miller, G. A. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343-352. doi:10.1037/0033-295x.101.2.343Pallett, P. M., Link, S., & Lee, K. (2010). New «golden» ratios for facial beauty. Vision Research, 50(2), 149-154. doi:10.1016/j.visres.2009.11.003Paunonen, S. V., Ewan, K., Earthy, J., Lefave, S., & Goldberg, H. (1999). Facial Features as Personality Cues. Journal of Personality, 67(3), 555-583. doi:10.1111/1467-6494.00065Petrican, R., Todorov, A., & Grady, C. (2014). Personality at Face Value: Facial Appearance Predicts Self and Other Personality Judgments among Strangers and Spouses. Journal of Nonverbal Behavior, 38(2), 259-277. doi:10.1007/s10919-014-0175-3Piepers, D. W., & Robbins, R. A. (2012). A Review and Clarification of the Terms «holistic,» «configural,» and «relational» in the Face Perception Literature. Frontiers in Psychology, 3. doi:10.3389/fpsyg.2012.00559Rakover, S. S. (2002). Featural vs. configurational information in faces: A conceptual and empirical analysis. British Journal of Psychology, 93(1), 1-30. doi:10.1348/000712602162427Rhodes, G., Ewing, L., Hayward, W. G., Maurer, D., Mondloch, C. J., & Tanaka, J. W. (2009). Contact and other-race effects in configural and component processing of faces. British Journal of Psychology, 100(4), 717-728. doi:10.1348/000712608x396503Richardson, J. T. E. (2011). Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review, 6(2), 135-147. doi:10.1016/j.edurev.2010.12.001Ritz-Timme, S., Gabriel, P., Obertovà, Z., Boguslawski, M., Mayer, F., Drabik, A., … Cattaneo, C. (2010). A new atlas for the evaluation of facial features: advantages, limits, and applicability. International Journal of Legal Medicine, 125(2), 301-306. doi:10.1007/s00414-010-0446-4Rojas Q., M., Masip, D., Todorov, A., & Vitria, J. (2011). Automatic Prediction of Facial Trait Judgments: Appearance vs. Structural Models. PLoS ONE, 6(8), e23323. doi:10.1371/journal.pone.0023323Rossion, B. (2008). Picture-plane inversion leads to qualitative changes of face perception. Acta Psychologica, 128(2), 274-289. doi:10.1016/j.actpsy.2008.02.003Russell, R. (2003). Sex, Beauty, and the Relative Luminance of Facial Features. Perception, 32(9), 1093-1107. doi:10.1068/p5101Saavedra, C., Smith, P., & Peissig, J. (2013). The Relative Role of Eyes, Eyebrows, and Eye Region in Face Recognition. Journal of Vision, 13(9), 410-410. doi:10.1167/13.9.410Sadr, J., Jarudi, I., & Sinha, P. (2003). The Role of Eyebrows in Face Recognition. Perception, 32(3), 285-293. doi:10.1068/p5027Said, C., Sebe, N., & Todorov, A. (2009). «Structural resemblance to emotional expressions predicts evaluation of emotionally neutral faces»: Correction to Said, Sebe, and Todorov (2009). Emotion, 9(4), 509-509. doi:10.1037/a0016784Scharff, A., Palmer, J., & Moore, C. M. (2011). Evidence of fixed capacity in visual object categorization. Psychonomic Bulletin & Review, 18(4), 713-721. doi:10.3758/s13423-011-0101-1Schobert, A.-K., Corradi-Dell’Acqua, C., Frühholz, S., van der Zwaag, W., & Vuilleumier, P. (2017). Functional organization of face processing in the human superior temporal sulcus: a 7T high-resolution fMRI study. Social Cognitive and Affective Neuroscience, 13(1), 102-113. doi:10.1093/scan/nsx119Sirovich, L., & Kirby, M. (1987). Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, 4(3), 519. doi:10.1364/josaa.4.000519Tanaka, J. W., & Farah, M. J. (1993). Parts and Wholes in Face Recognition. The Quarterly Journal of Experimental Psychology Section A, 46(2), 225-245. doi:10.1080/14640749308401045Taubert, J., Apthorp, D., Aagten-Murphy, D., & Alais, D. (2011). The role of holistic processing in face perception: Evidence from the face inversion effect. Vision Research, 51(11), 1273-1278. doi:10.1016/j.visres.2011.04.002Terry, R. L. (1977). Further Evidence on Components of Facial Attractiveness. Perceptual and Motor Skills, 45(1), 130-130. doi:10.2466/pms.1977.45.1.130Todorov, A., Dotsch, R., Wigboldus, D. H. J., & Said, C. P. (2011). Data-driven Methods for Modeling Social Perception. Social and Personality Psychology Compass, 5(10), 775-791. doi:10.1111/j.1751-9004.2011.00389.xTodorov, A., Mandisodza, A. N., Goren, A., & Hall, C. C. (2005). Inferences of Competence from Faces Predict Election Outcomes. Science, 308(5728), 1623-1626. doi:10.1126/science.1110589Todorov, A., Said, C. P., Engell, A. D., & Oosterhof, N. N. (2008). Understanding evaluation of faces on social dimensions. Trends in Cognitive Sciences, 12(12), 455-460. doi:10.1016/j.tics.2008.10.001Tsankova, E., & Kappas, A. (2015). Facial Skin Smoothness as an Indicator of Perceived Trustworthiness and Related Traits. Perception, 45(4), 400-408. doi:10.1177/0301006615616748Turk, M., & Pentland, A. (1991). Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 3(1), 71-86. doi:10.1162/jocn.1991.3.1.71Wang, R., Li, J., Fang, H., Tian, M., & Liu, J. (2012). Individual Differences in Holistic Processing Predict Face Recognition Ability. Psychological Science, 23(2), 169-177. doi:10.1177/0956797611420575Wilson, J. P., & Rule, N. O. (2015). Facial Trustworthiness Predicts Extreme Criminal-Sentencing Outcomes. Psychological Science, 26(8), 1325-1331. doi:10.1177/0956797615590992Yamaguchi, M. K., Hirukawa, T., & Kanazawa, S. (2013). Judgment of Gender through Facial Parts. Perception, 42(11), 1253-1265. doi:10.1068/p240563nMcArthur, L. Z., & Baron, R. M. (1983). Toward an ecological theory of social perception. Psychological Review, 90(3), 215-238. doi:10.1037/0033-295x.90.3.21