The Influence of Each Facial Feature on How We Perceive and Interpret Human Faces

[EN] Facial information is processed by our brain in such a way that we immediately make judgments about, for example, attractiveness or masculinity or interpret personality traits or moods of other people. The appearance of each facial feature has an effect on our perception of facial traits. This research addresses the problem of measuring the size of these effects for five facial features (eyes, eyebrows, nose, mouth, and jaw). Our proposal is a mixed feature-based and image-based approach that allows judgments to be made on complete real faces in the categorization tasks, more than on synthetic, noisy, or partial faces that can influence the assessment. Each facial feature of the faces is automatically classified considering their global appearance using principal component analysis. Using this procedure, we establish a reduced set of relevant specific attributes (each one describing a complete facial feature) to characterize faces. In this way, a more direct link can be established between perceived facial traits and what people intuitively consider an eye, an eyebrow, a nose, a mouth, or a jaw. A set of 92 male faces were classified using this procedure, and the results were related to their scores in 15 perceived facial traits. We show that the relevant features greatly depend on what we are trying to judge. Globally, the eyes have the greatest effect. However, other facial features are more relevant for some judgments like the mouth for happiness and femininity or the nose for dominance.This study was carried out using the Chicago Face Database developed at the University of Chicago by Debbie S. Ma, Joshua Correll, and Bernd Wittenbrink.Diego-Mas, JA.; Fuentes-Hurtado, FJ.; Naranjo Ornedo, V.; Alcañiz Raya, ML. (2020). The Influence of Each Facial Feature on How We Perceive and Interpret Human Faces. i-Perception. 11(5):1-18. https://doi.org/10.1177/2041669520961123S118115Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face Description with Local Binary Patterns: Application to Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2037-2041. doi:10.1109/tpami.2006.244Axelrod, V., & Yovel, G. (2010). External facial features modify the representation of internal facial features in the fusiform face area. NeuroImage, 52(2), 720-725. doi:10.1016/j.neuroimage.2010.04.027Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711-720. doi:10.1109/34.598228Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94(2), 115-147. doi:10.1037/0033-295x.94.2.115Blais, C., Roy, C., Fiset, D., Arguin, M., & Gosselin, F. (2012). The eyes are not the window to basic emotions. Neuropsychologia, 50(12), 2830-2838. doi:10.1016/j.neuropsychologia.2012.08.010Bovet, J., Barthes, J., Durand, V., Raymond, M., & Alvergne, A. (2012). Men’s Preference for Women’s Facial Features: Testing Homogamy and the Paternity Uncertainty Hypothesis. PLoS ONE, 7(11), e49791. doi:10.1371/journal.pone.0049791Brahnam, S., & Nanni, L. (2010). Predicting trait impressions of faces using local face recognition techniques. Expert Systems with Applications, 37(7), 5086-5093. doi:10.1016/j.eswa.2009.12.002Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77(3), 305-327. doi:10.1111/j.2044-8295.1986.tb02199.xCabeza, R., & Kato, T. (2000). Features are Also Important: Contributions of Featural and Configural Processing to Face Recognition. Psychological Science, 11(5), 429-433. doi:10.1111/1467-9280.00283Chihaoui, M., Elkefi, A., Bellil, W., & Ben Amar, C. (2016). A Survey of 2D Face Recognition Techniques. Computers, 5(4), 21. doi:10.3390/computers5040021Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681-685. doi:10.1109/34.927467Diamond, R., & Carey, S. (1986). Why faces are and are not special: An effect of expertise. Journal of Experimental Psychology: General, 115(2), 107-117. doi:10.1037/0096-3445.115.2.107Dixson, B. J. W., Sulikowski, D., Gouda‐Vossos, A., Rantala, M. J., & Brooks, R. C. (2016). The masculinity paradox: facial masculinity and beardedness interact to determine women’s ratings of men’s facial attractiveness. Journal of Evolutionary Biology, 29(11), 2311-2320. doi:10.1111/jeb.12958Dunn†, J. C. (1974). Well-Separated Clusters and Optimal Fuzzy Partitions. Journal of Cybernetics, 4(1), 95-104. doi:10.1080/01969727408546059Eberhardt, J. L., Davies, P. G., Purdie-Vaughns, V. J., & Johnson, S. L. (2006). Looking Deathworthy. Psychological Science, 17(5), 383-386. doi:10.1111/j.1467-9280.2006.01716.xFink, B., Neave, N., Manning, J. T., & Grammer, K. (2006). Facial symmetry and judgements of attractiveness, health and personality. Personality and Individual Differences, 41(3), 491-499. doi:10.1016/j.paid.2006.01.017Fox, E., & Damjanovic, L. (2006). The eyes are sufficient to produce a threat superiority effect. Emotion, 6(3), 534-539. doi:10.1037/1528-3542.6.3.534Fuentes-Hurtado, F., Diego-Mas, J. A., Naranjo, V., & Alcañiz, M. (2019). Automatic classification of human facial features based on their appearance. PLOS ONE, 14(1), e0211314. doi:10.1371/journal.pone.0211314Gill, D. (2017). Women and men integrate facial information differently in appraising the beauty of a face. Evolution and Human Behavior, 38(6), 756-760. doi:10.1016/j.evolhumbehav.2017.07.001Gosselin, F., & Schyns, P. G. (2001). Bubbles: a technique to reveal the use of information in recognition tasks. Vision Research, 41(17), 2261-2271. doi:10.1016/s0042-6989(01)00097-9Hagiwara, N., Kashy, D. A., & Cesario, J. (2012). The independent effects of skin tone and facial features on Whites’ affective reactions to Blacks. Journal of Experimental Social Psychology, 48(4), 892-898. doi:10.1016/j.jesp.2012.02.001Hayward, W. G., Rhodes, G., & Schwaninger, A. (2008). An own-race advantage for components as well as configurations in face recognition. Cognition, 106(2), 1017-1027. doi:10.1016/j.cognition.2007.04.002Jack, R. E., & Schyns, P. G. (2015). The Human Face as a Dynamic Tool for Social Communication. Current Biology, 25(14), R621-R634. doi:10.1016/j.cub.2015.05.052Jones, B. C., Little, A. C., Burt, D. M., & Perrett, D. I. (2004). When Facial Attractiveness is Only Skin Deep. Perception, 33(5), 569-576. doi:10.1068/p3463Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception. The Journal of Neuroscience, 17(11), 4302-4311. doi:10.1523/jneurosci.17-11-04302.1997Keating, C. F., & Doyle, J. (2002). The faces of desirable mates and dates contain mixed social status cues. Journal of Experimental Social Psychology, 38(4), 414-424. doi:10.1016/s0022-1031(02)00007-0Keil, M. S. (2009). «I Look in Your Eyes, Honey»: Internal Face Features Induce Spatial Frequency Preference for Human Face Processing. PLoS Computational Biology, 5(3), e1000329. doi:10.1371/journal.pcbi.1000329Kwart, D. G., Foulsham, T., & Kingstone, A. (2012). Age and Beauty are in the Eye of the Beholder. Perception, 41(8), 925-938. doi:10.1068/p7136Langlois, J. H., Kalakanis, L., Rubenstein, A. J., Larson, A., Hallam, M., & Smoot, M. (2000). Maxims or myths of beauty? A meta-analytic and theoretical review. Psychological Bulletin, 126(3), 390-423. doi:10.1037/0033-2909.126.3.390Levine, T. R., & Hullett, C. R. (2002). Eta Squared, Partial Eta Squared, and Misreporting of Effect Size in Communication Research. Human Communication Research, 28(4), 612-625. doi:10.1111/j.1468-2958.2002.tb00828.xLittle, A. C., Burriss, R. P., Jones, B. C., & Roberts, S. C. (2007). Facial appearance affects voting decisions. Evolution and Human Behavior, 28(1), 18-27. doi:10.1016/j.evolhumbehav.2006.09.002Lundqvist, D., Esteves, F., & Ohman, A. (1999). The Face of Wrath: Critical Features for Conveying Facial Threat. Cognition & Emotion, 13(6), 691-711. doi:10.1080/026999399379041Ma, D. S., Correll, J., & Wittenbrink, B. (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior Research Methods, 47(4), 1122-1135. doi:10.3758/s13428-014-0532-5Maloney, L. T., & Dal Martello, M. F. (2006). Kin recognition and the perceived facial similarity of children. Journal of Vision, 6(10), 4. doi:10.1167/6.10.4McKone, E., & Yovel, G. (2009). Why does picture-plane inversion sometimes dissociate perception of features and spacing in faces, and sometimes not? Toward a new theory of holistic processing. Psychonomic Bulletin & Review, 16(5), 778-797. doi:10.3758/pbr.16.5.778Meyers, E., & Wolf, L. (2007). Using Biologically Inspired Features for Face Processing. International Journal of Computer Vision, 76(1), 93-104. doi:10.1007/s11263-007-0058-8Miller, G. A. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343-352. doi:10.1037/0033-295x.101.2.343Pallett, P. M., Link, S., & Lee, K. (2010). New «golden» ratios for facial beauty. Vision Research, 50(2), 149-154. doi:10.1016/j.visres.2009.11.003Paunonen, S. V., Ewan, K., Earthy, J., Lefave, S., & Goldberg, H. (1999). Facial Features as Personality Cues. Journal of Personality, 67(3), 555-583. doi:10.1111/1467-6494.00065Petrican, R., Todorov, A., & Grady, C. (2014). Personality at Face Value: Facial Appearance Predicts Self and Other Personality Judgments among Strangers and Spouses. Journal of Nonverbal Behavior, 38(2), 259-277. doi:10.1007/s10919-014-0175-3Piepers, D. W., & Robbins, R. A. (2012). A Review and Clarification of the Terms «holistic,» «configural,» and «relational» in the Face Perception Literature. Frontiers in Psychology, 3. doi:10.3389/fpsyg.2012.00559Rakover, S. S. (2002). Featural vs. configurational information in faces: A conceptual and empirical analysis. British Journal of Psychology, 93(1), 1-30. doi:10.1348/000712602162427Rhodes, G., Ewing, L., Hayward, W. G., Maurer, D., Mondloch, C. J., & Tanaka, J. W. (2009). Contact and other-race effects in configural and component processing of faces. British Journal of Psychology, 100(4), 717-728. doi:10.1348/000712608x396503Richardson, J. T. E. (2011). Eta squared and partial eta squared as measures of effect size in educational research. Educational Research Review, 6(2), 135-147. doi:10.1016/j.edurev.2010.12.001Ritz-Timme, S., Gabriel, P., Obertovà, Z., Boguslawski, M., Mayer, F., Drabik, A., … Cattaneo, C. (2010). A new atlas for the evaluation of facial features: advantages, limits, and applicability. International Journal of Legal Medicine, 125(2), 301-306. doi:10.1007/s00414-010-0446-4Rojas Q., M., Masip, D., Todorov, A., & Vitria, J. (2011). Automatic Prediction of Facial Trait Judgments: Appearance vs. Structural Models. PLoS ONE, 6(8), e23323. doi:10.1371/journal.pone.0023323Rossion, B. (2008). Picture-plane inversion leads to qualitative changes of face perception. Acta Psychologica, 128(2), 274-289. doi:10.1016/j.actpsy.2008.02.003Russell, R. (2003). Sex, Beauty, and the Relative Luminance of Facial Features. Perception, 32(9), 1093-1107. doi:10.1068/p5101Saavedra, C., Smith, P., & Peissig, J. (2013). The Relative Role of Eyes, Eyebrows, and Eye Region in Face Recognition. Journal of Vision, 13(9), 410-410. doi:10.1167/13.9.410Sadr, J., Jarudi, I., & Sinha, P. (2003). The Role of Eyebrows in Face Recognition. Perception, 32(3), 285-293. doi:10.1068/p5027Said, C., Sebe, N., & Todorov, A. (2009). «Structural resemblance to emotional expressions predicts evaluation of emotionally neutral faces»: Correction to Said, Sebe, and Todorov (2009). Emotion, 9(4), 509-509. doi:10.1037/a0016784Scharff, A., Palmer, J., & Moore, C. M. (2011). Evidence of fixed capacity in visual object categorization. Psychonomic Bulletin & Review, 18(4), 713-721. doi:10.3758/s13423-011-0101-1Schobert, A.-K., Corradi-Dell’Acqua, C., Frühholz, S., van der Zwaag, W., & Vuilleumier, P. (2017). Functional organization of face processing in the human superior temporal sulcus: a 7T high-resolution fMRI study. Social Cognitive and Affective Neuroscience, 13(1), 102-113. doi:10.1093/scan/nsx119Sirovich, L., & Kirby, M. (1987). Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, 4(3), 519. doi:10.1364/josaa.4.000519Tanaka, J. W., & Farah, M. J. (1993). Parts and Wholes in Face Recognition. The Quarterly Journal of Experimental Psychology Section A, 46(2), 225-245. doi:10.1080/14640749308401045Taubert, J., Apthorp, D., Aagten-Murphy, D., & Alais, D. (2011). The role of holistic processing in face perception: Evidence from the face inversion effect. Vision Research, 51(11), 1273-1278. doi:10.1016/j.visres.2011.04.002Terry, R. L. (1977). Further Evidence on Components of Facial Attractiveness. Perceptual and Motor Skills, 45(1), 130-130. doi:10.2466/pms.1977.45.1.130Todorov, A., Dotsch, R., Wigboldus, D. H. J., & Said, C. P. (2011). Data-driven Methods for Modeling Social Perception. Social and Personality Psychology Compass, 5(10), 775-791. doi:10.1111/j.1751-9004.2011.00389.xTodorov, A., Mandisodza, A. N., Goren, A., & Hall, C. C. (2005). Inferences of Competence from Faces Predict Election Outcomes. Science, 308(5728), 1623-1626. doi:10.1126/science.1110589Todorov, A., Said, C. P., Engell, A. D., & Oosterhof, N. N. (2008). Understanding evaluation of faces on social dimensions. Trends in Cognitive Sciences, 12(12), 455-460. doi:10.1016/j.tics.2008.10.001Tsankova, E., & Kappas, A. (2015). Facial Skin Smoothness as an Indicator of Perceived Trustworthiness and Related Traits. Perception, 45(4), 400-408. doi:10.1177/0301006615616748Turk, M., & Pentland, A. (1991). Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 3(1), 71-86. doi:10.1162/jocn.1991.3.1.71Wang, R., Li, J., Fang, H., Tian, M., & Liu, J. (2012). Individual Differences in Holistic Processing Predict Face Recognition Ability. Psychological Science, 23(2), 169-177. doi:10.1177/0956797611420575Wilson, J. P., & Rule, N. O. (2015). Facial Trustworthiness Predicts Extreme Criminal-Sentencing Outcomes. Psychological Science, 26(8), 1325-1331. doi:10.1177/0956797615590992Yamaguchi, M. K., Hirukawa, T., & Kanazawa, S. (2013). Judgment of Gender through Facial Parts. Perception, 42(11), 1253-1265. doi:10.1068/p240563nMcArthur, L. Z., & Baron, R. M. (1983). Toward an ecological theory of social perception. Psychological Review, 90(3), 215-238. doi:10.1037/0033-295x.90.3.21

Using latent features for short-term person re-identification with RGB-D cameras

This paper presents a system for people re-identification in uncontrolled scenarios using RGB-depth cameras. Compared to conventional RGB cameras, the use of depth information greatly simplifies the tasks of segmentation and tracking. In a previous work, we proposed a similar architecture where people were characterized using color-based descriptors that we named bodyprints. In this work, we propose the use of latent feature models to extract more relevant information from the bodyprint descriptors by reducing their dimensionality. Latent features can also cope with missing data in case of occlusions. Different probabilistic latent feature models, such as probabilistic principal component analysis and factor analysis, are compared in the paper. The main difference between the models is how the observation noise is handled in each case. Re-identification experiments have been conducted in a real store where people behaved naturally. The results show that the use of the latent features significantly improves the re-identification rates compared to state-of-the-art works.The work presented in this paper has been funded by the Spanish Ministry of Science and Technology under the CICYT contract TEVISMART, TEC2009-09146.Oliver Moll, J.; Albiol Colomer, A.; Albiol Colomer, AJ.; Mossi García, JM. (2016). Using latent features for short-term person re-identification with RGB-D cameras. Pattern Analysis and Applications. 19(2):549-561. https://doi.org/10.1007/s10044-015-0489-8S549561192http://kinectforwindows.org/http://www.gpiv.upv.es/videoresearch/personindexing.htmlAlbiol A, Albiol A, Oliver J, Mossi JM (2012) Who is who at different cameras. Matching people using depth cameras. Comput Vis IET 6(5):378–387Bak S, Corvee E, Bremond F, Thonnat M (2010) Person re-identification using haar-based and dcd-based signature. In: 2nd workshop on activity monitoring by multi-camera surveillance systems, AMMCSS 2010, in conjunction with 7th IEEE international conference on advanced video and signal-based surveillance, AVSS. AVSSBak S, Corvee E, Bremond F, Thonnat M (2010) Person re-identification using spatial covariance regions of human body parts. In: Seventh IEEE international conference on advanced video and signal based surveillance. pp. 435–440Bak S, Corvee E, Bremond F, Thonnat M (2011) Multiple-shot human re-identification by mean riemannian covariance grid. In: Advanced video and signal-based surveillance. Klagenfurt, Autriche. http://hal.inria.fr/inria-00620496Baltieri D, Vezzani R, Cucchiara R, Utasi A, BenedeK C, Szirányi T (2011) Multi-view people surveillance using 3d information. In: ICCV workshops. pp. 1817–1824Barbosa BI, Cristani M, Del Bue A, Bazzani L, Murino V (2012) Re-identification with rgb-d sensors. In: First international workshop on re-identificationBasilevsky A (1994) Statistical factor analysis and related methods: theory and applications. Willey, New YorkBäuml M, Bernardin K, Fischer k, Ekenel HK, Stiefelhagen R (2010) Multi-pose face recognition for person retrieval in camera networks. In: International conference on advanced video and signal-based surveillanceBazzani L, Cristani M, Perina A, Farenzena M, Murino V (2010) Multiple-shot person re-identification by hpe signature. In: Proceedings of the 2010 20th international conference on pattern recognition. Washington, DC, USA, pp. 1413–1416Bird ND, Masoud O, Papanikolopoulos NP, Isaacs A (2005) Detection of loitering individuals in public transportation areas. IEEE Trans Intell Transp Syst 6(2):167–177Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, SecaucusCha SH (2007) Comprehensive survey on distance/similarity measures between probability density functions. Int J Math Models Methods Appl Sci 1(4):300–307Cheng YM, Zhou WT, Wang Y, Zhao CH, Zhang SW (2009) Multi-camera-based object handoff using decision-level fusion. In: Conference on image and signal processing. pp. 1–5Dikmen M, Akbas E, Huang TS, Ahuja N (2010) Pedestrian recognition with a learned metric. In: Asian conference in computer visionDoretto G, Sebastian T, Tu P, Rittscher J (2011) Appearance-based person reidentification in camera networks: problem overview and current approaches. J Ambient Intell Humaniz Comput 2:1–25Farenzena M, Bazzani L, Perina A, Murino V, Cristani M (2010) Person re-identification by symmetry-driven accumulation of local features. In: Proceedings of the 2010 IEEE computer society conference on computer vision and pattern recognition (CVPR 2010). IEEE Computer Society, San Francisco, CA, USAFodor I (2002) A survey of dimension reduction techniques. Technical report. Lawrence Livermore National LaboratoryFreund Y, Iyer R, Schapire RE, Singer Y (2003) An efficient boosting algorithm for combining preferences. J Mach Learn Res 4:933–969Gandhi T, Trivedi M (2006) Panoramic appearance map (pam) for multi-camera based person re-identification. Advanced Video and Signal Based Surveillance, IEEE Conference on, p. 78Garcia J, Gardel A, Bravo I, Lazaro J (2014) Multiple view oriented matching algorithm for people reidentification. Ind Inform IEEE Trans 10(3):1841–1851Gheissari N, Sebastian TB, Hartley R (2006) Person reidentification using spatiotemporal appearance. CVPR 2:1528–1535Gray D, Brennan S, Tao H (2007) Evaluating appearance models for recognition, reacquisition, and tracking. In: Proceedings of IEEE international workshop on performance evaluation for tracking and surveillance (PETS)Gray D, Tao H (2008) Viewpoint invariant pedestrian recognition with an ensemble of localized features. In: Proceedings of the 10th european conference on computer vision: part I. Berlin, pp. 262–275 (2008)Ilin A, Raiko T (2010) Practical approaches to principal component analysis in the presence of missing values. J Mach Learn Res 99:1957–2000Javed O, Shafique O, Rasheed Z, Shah M (2008) Modeling inter-camera space–time and appearance relationships for tracking across non-overlapping views. Comput Vis Image Underst 109(2):146–162Kai J, Bodensteiner C, Arens M (2011) Person re-identification in multi-camera networks. In: Computer vision and pattern recognition workshops (CVPRW), 2011 IEEE computer society conference on, pp. 55–61Kuo CH, Huang C, Nevatia R (2010) Inter-camera association of multi-target tracks by on-line learned appearance affinity models. Proceedings of the 11th european conference on computer vision: part I, ECCV’10. Springer, Berlin, pp 383–396Lan R, Zhou Y, Tang YY, Chen C (2014) Person reidentification using quaternionic local binary pattern. In: Multimedia and expo (ICME), 2014 IEEE international conference on, pp. 1–6Loy CC, Liu C, Gong S (2013) Person re-identification by manifold ranking. In: icip. pp. 3318–3325Madden C, Cheng E, Piccardi M (2007) Tracking people across disjoint camera views by an illumination-tolerant appearance representation. Mach Vis Appl 18:233–247Mazzon R, Tahir SF, Cavallaro A (2012) Person re-identification in crowd. Pattern Recogn Lett 33(14):1828–1837Oliveira IO, Souza Pio JL (2009) People reidentification in a camera network. In: Eighth IEEE international conference on dependable, autonomic and secure computing. pp. 461–466Papadakis P, Pratikakis I, Theoharis T, Perantonis SJ (2010) Panorama: a 3d shape descriptor based on panoramic views for unsupervised 3d object retrieval. Int J Comput Vis 89(2–3):177–192Prosser B, Zheng WS, Gong S, Xiang T (2010) Person re-identification by support vector ranking. In: Proceedings of the British machine vision conference. BMVA Press, pp. 21.1–21.11Roweis S (1998) Em algorithms for pca and spca. In: Advances in neural information processing systems. MIT Press, Cambridge, pp. 626–632 (1998)Pedagadi S, Orwell J, Velastin S, Boghossian B (2013) Local fisher discriminant analysis for pedestrian re-identification. In: CVPR. pp. 3318–3325Satta R, Fumera G, Roli F (2012) Fast person re-identification based on dissimilarity representations. Pattern Recogn Lett, Special Issue on Novel Pattern Recognition-Based Methods for Reidentification in Biometric Context 33:1838–1848Tao D, Jin L, Wang Y, Li X (2015) Person reidentification by minimum classification error-based kiss metric learning. Cybern IEEE Trans 45(2):242–252Tipping ME, Bishop CM (1999) Probabilistic principal component analysis. J R Stat Soc Ser B 61:611–622Tisse CL, Martin L, Torres L, Robert M (2002) Person identification technique using human iris recognition. In: Proceedings of vision interface, pp 294–299Vandergheynst P, Bierlaire M, Kunt M, Alahi A (2009) Cascade of descriptors to detect and track objects across any network of cameras. Comput Vis Image Underst, pp 1413–1416Verbeek J (2009) Notes on probabilistic pca with missing values. Technical reportWang D, Chen CO, Chen TY, Lee CT (2009) People recognition for entering and leaving a video surveillance area. In: Fourth international conference on innovative computing, information and control. pp. 334–337Zhang Z, Troje NF (2005) View-independent person identification from human gait. Neurocomputing 69:250–256Zhao T, Aggarwal M, Kumar R, Sawhney H (2005) Real-time wide area multi-camera stereo tracking. In: IEEE computer society conference on computer vision and pattern recognition. pp. 976–983Zheng S, Xie B, Huang K, Tao D (2011) Multi-view pedestrian recognition using shared dictionary learning with group sparsity. In: Lu BL, Zhang L, Kwok JT (eds) ICONIP (3), Lecture notes in computer science, vol 7064. Springer, New York, pp. 629–638Zheng WS, Gong S, Xiang T (2011) Person re-identification by probabilistic relative distance comparison. In: Computer vision and pattern recognition (CVPR), 2011 IEEE conference on. pp. 649–65

Automatic classification of human facial features based on their appearance

[EN] Classification or typology systems used to categorize different human body parts have existed for many years. Nevertheless, there are very few taxonomies of facial features. Ergonomics, forensic anthropology, crime prevention or new human-machine interaction systems and online activities, like e-commerce, e-learning, games, dating or social networks, are fields in which classifications of facial features are useful, for example, to create digital interlocutors that optimize the interactions between human and machines. However, classifying isolated facial features is difficult for human observers. Previous works reported low inter-observer and intra-observer agreement in the evaluation of facial features. This work presents a computer-based procedure to automatically classify facial features based on their global appearance. This procedure deals with the difficulties associated with classifying features using judgements from human observers, and facilitates the development of taxonomies of facial features. Taxonomies obtained through this procedure are presented for eyes, mouths and noses.Fuentes-Hurtado, F.; Diego-Mas, JA.; Naranjo Ornedo, V.; Alcañiz Raya, ML. (2019). Automatic classification of human facial features based on their appearance. PLoS ONE. 14(1):1-20. https://doi.org/10.1371/journal.pone.0211314S120141Damasio, A. R. (1985). Prosopagnosia. Trends in Neurosciences, 8, 132-135. doi:10.1016/0166-2236(85)90051-7Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77(3), 305-327. doi:10.1111/j.2044-8295.1986.tb02199.xTodorov, A. (2011). Evaluating Faces on Social Dimensions. Social Neuroscience, 54-76. doi:10.1093/acprof:oso/9780195316872.003.0004Little, A. C., Burriss, R. P., Jones, B. C., & Roberts, S. C. (2007). Facial appearance affects voting decisions. Evolution and Human Behavior, 28(1), 18-27. doi:10.1016/j.evolhumbehav.2006.09.002Porter, J. P., & Olson, K. L. (2001). Anthropometric Facial Analysis of the African American Woman. Archives of Facial Plastic Surgery, 3(3), 191-197. doi:10.1001/archfaci.3.3.191Gündüz Arslan, S., Genç, C., Odabaş, B., & Devecioğlu Kama, J. (2007). Comparison of Facial Proportions and Anthropometric Norms Among Turkish Young Adults With Different Face Types. Aesthetic Plastic Surgery, 32(2), 234-242. doi:10.1007/s00266-007-9049-yFerring, V., & Pancherz, H. (2008). Divine proportions in the growing face. American Journal of Orthodontics and Dentofacial Orthopedics, 134(4), 472-479. doi:10.1016/j.ajodo.2007.03.027Mane, D. R., Kale, A. D., Bhai, M. B., & Hallikerimath, S. (2010). Anthropometric and anthroposcopic analysis of different shapes of faces in group of Indian population: A pilot study. Journal of Forensic and Legal Medicine, 17(8), 421-425. doi:10.1016/j.jflm.2010.09.001Ritz-Timme, S., Gabriel, P., Tutkuviene, J., Poppa, P., Obertová, Z., Gibelli, D., … Cattaneo, C. (2011). Metric and morphological assessment of facial features: A study on three European populations. Forensic Science International, 207(1-3), 239.e1-239.e8. doi:10.1016/j.forsciint.2011.01.035Ritz-Timme, S., Gabriel, P., Obertovà, Z., Boguslawski, M., Mayer, F., Drabik, A., … Cattaneo, C. (2010). A new atlas for the evaluation of facial features: advantages, limits, and applicability. International Journal of Legal Medicine, 125(2), 301-306. doi:10.1007/s00414-010-0446-4Kong, S. G., Heo, J., Abidi, B. R., Paik, J., & Abidi, M. A. (2005). Recent advances in visual and infrared face recognition—a review. Computer Vision and Image Understanding, 97(1), 103-135. doi:10.1016/j.cviu.2004.04.001Tavares, G., Mourão, A., & Magalhães, J. (2016). Crowdsourcing facial expressions for affective-interaction. Computer Vision and Image Understanding, 147, 102-113. doi:10.1016/j.cviu.2016.02.001Buckingham, G., DeBruine, L. M., Little, A. C., Welling, L. L. M., Conway, C. A., Tiddeman, B. P., & Jones, B. C. (2006). Visual adaptation to masculine and feminine faces influences generalized preferences and perceptions of trustworthiness. Evolution and Human Behavior, 27(5), 381-389. doi:10.1016/j.evolhumbehav.2006.03.001Boberg M, Piippo P, Ollila E. Designing Avatars. DIMEA ‘08 Proc 3rd Int Conf Digit Interact Media Entertain Arts. ACM; 2008; 232–239. doi: https://doi.org/10.1145/1413634.1413679Rojas Q., M., Masip, D., Todorov, A., & Vitria, J. (2011). Automatic Prediction of Facial Trait Judgments: Appearance vs. Structural Models. PLoS ONE, 6(8), e23323. doi:10.1371/journal.pone.0023323Laurentini, A., & Bottino, A. (2014). Computer analysis of face beauty: A survey. Computer Vision and Image Understanding, 125, 184-199. doi:10.1016/j.cviu.2014.04.006Alemany S, Gonzalez J, Nacher B, Soriano C, Arnaiz C, Heras H. Anthropometric survey of the Spanish female population aimed at the apparel industry. Proceedings of the 2010 Intl Conference on 3D Body scanning Technologies. 2010. pp. 307–315.Vinué, G., Epifanio, I., & Alemany, S. (2015). Archetypoids: A new approach to define representative archetypal data. Computational Statistics & Data Analysis, 87, 102-115. doi:10.1016/j.csda.2015.01.018Jee, S., & Yun, M. H. (2016). An anthropometric survey of Korean hand and hand shape types. International Journal of Industrial Ergonomics, 53, 10-18. doi:10.1016/j.ergon.2015.10.004Kim, N.-S., & Do, W.-H. (2014). Classification of Elderly Women’s Foot Type. Journal of the Korean Society of Clothing and Textiles, 38(3), 305-320. doi:10.5850/jksct.2014.38.3.305Sarakon P, Charoenpong T, Charoensiriwath S. Face shape classification from 3D human data by using SVM. The 7th 2014 Biomedical Engineering International Conference. IEEE; 2014. pp. 1–5. doi: https://doi.org/10.1109/BMEiCON.2014.7017382PRESTON, T. A., & SINGH, M. (1972). Redintegrated Somatotyping. Ergonomics, 15(6), 693-700. doi:10.1080/00140137208924469Lin, Y.-L., & Lee, K.-L. (1999). Investigation of anthropometry basis grouping technique for subject classification. Ergonomics, 42(10), 1311-1316. doi:10.1080/001401399184965Malousaris, G. G., Bergeles, N. K., Barzouka, K. G., Bayios, I. A., Nassis, G. P., & Koskolou, M. D. (2008). Somatotype, size and body composition of competitive female volleyball players. Journal of Science and Medicine in Sport, 11(3), 337-344. doi:10.1016/j.jsams.2006.11.008Carvalho, P. V. R., dos Santos, I. L., Gomes, J. O., Borges, M. R. S., & Guerlain, S. (2008). Human factors approach for evaluation and redesign of human–system interfaces of a nuclear power plant simulator. Displays, 29(3), 273-284. doi:10.1016/j.displa.2007.08.010Fabri M, Moore D. The use of emotionally expressive avatars in Collaborative Virtual Environments. AISB’05 Convention:Proceedings of the Joint Symposium on Virtual Social Agents: Social Presence Cues for Virtual Humanoids Empathic Interaction with Synthetic Characters Mind Minding Agents. 2005. pp. 88–94. doi:citeulike-article-id:790934Sukhija, P., Behal, S., & Singh, P. (2016). Face Recognition System Using Genetic Algorithm. Procedia Computer Science, 85, 410-417. doi:10.1016/j.procs.2016.05.183Trescak T, Bogdanovych A, Simoff S, Rodriguez I. Generating diverse ethnic groups with genetic algorithms. Proceedings of the 18th ACM symposium on Virtual reality software and technology—VRST ‘12. New York, New York, USA: ACM Press; 2012. p. 1. doi: https://doi.org/10.1145/2407336.2407338Vanezis, P., Lu, D., Cockburn, J., Gonzalez, A., McCombe, G., Trujillo, O., & Vanezis, M. (1996). Morphological Classification of Facial Features in Adult Caucasian Males Based on an Assessment of Photographs of 50 Subjects. Journal of Forensic Sciences, 41(5), 13998J. doi:10.1520/jfs13998jTamir, A. (2011). Numerical Survey of the Different Shapes of the Human Nose. Journal of Craniofacial Surgery, 22(3), 1104-1107. doi:10.1097/scs.0b013e3182108eb3Tamir, A. (2013). Numerical Survey of the Different Shapes of Human Chin. Journal of Craniofacial Surgery, 24(5), 1657-1659. doi:10.1097/scs.0b013e3182942b77Richler, J. J., Cheung, O. S., & Gauthier, I. (2011). Holistic Processing Predicts Face Recognition. Psychological Science, 22(4), 464-471. doi:10.1177/0956797611401753Taubert, J., Apthorp, D., Aagten-Murphy, D., & Alais, D. (2011). The role of holistic processing in face perception: Evidence from the face inversion effect. Vision Research, 51(11), 1273-1278. doi:10.1016/j.visres.2011.04.002Donnelly, N., & Davidoff, J. (1999). The Mental Representations of Faces and Houses: Issues Concerning Parts and Wholes. Visual Cognition, 6(3-4), 319-343. doi:10.1080/135062899395000Davidoff, J., & Donnelly, N. (1990). Object superiority: A comparison of complete and part probes. Acta Psychologica, 73(3), 225-243. doi:10.1016/0001-6918(90)90024-aTanaka, J. W., & Farah, M. J. (1993). Parts and Wholes in Face Recognition. The Quarterly Journal of Experimental Psychology Section A, 46(2), 225-245. doi:10.1080/14640749308401045Wang, R., Li, J., Fang, H., Tian, M., & Liu, J. (2012). Individual Differences in Holistic Processing Predict Face Recognition Ability. Psychological Science, 23(2), 169-177. doi:10.1177/0956797611420575Rhodes, G., Ewing, L., Hayward, W. G., Maurer, D., Mondloch, C. J., & Tanaka, J. W. (2009). Contact and other-race effects in configural and component processing of faces. British Journal of Psychology, 100(4), 717-728. doi:10.1348/000712608x396503Miller, G. A. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343-352. doi:10.1037/0033-295x.101.2.343Scharff, A., Palmer, J., & Moore, C. M. (2011). Evidence of fixed capacity in visual object categorization. Psychonomic Bulletin & Review, 18(4), 713-721. doi:10.3758/s13423-011-0101-1Meyers, E., & Wolf, L. (2007). Using Biologically Inspired Features for Face Processing. International Journal of Computer Vision, 76(1), 93-104. doi:10.1007/s11263-007-0058-8Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681-685. doi:10.1109/34.927467Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face Description with Local Binary Patterns: Application to Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2037-2041. doi:10.1109/tpami.2006.244Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711-720. doi:10.1109/34.598228Turk, M., & Pentland, A. (1991). Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 3(1), 71-86. doi:10.1162/jocn.1991.3.1.71Klare B, Jain AK. On a taxonomy of facial features. IEEE 4th International Conference on Biometrics: Theory, Applications and Systems, BTAS 2010. IEEE; 2010. pp. 1–8. doi: https://doi.org/10.1109/BTAS.2010.5634533Chihaoui, M., Elkefi, A., Bellil, W., & Ben Amar, C. (2016). A Survey of 2D Face Recognition Techniques. Computers, 5(4), 21. doi:10.3390/computers5040021Ma, D. S., Correll, J., & Wittenbrink, B. (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior Research Methods, 47(4), 1122-1135. doi:10.3758/s13428-014-0532-5Asthana A, Zafeiriou S, Cheng S, Pantic M. Incremental face alignment in the wild. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2014. pp. 1859–1866. doi: https://doi.org/10.1109/CVPR.2014.240Bag S, Barik S, Sen P, Sanyal G. A statistical nonparametric approach of face recognition: combination of eigenface & modified k-means clustering. Proceedings Second International Conference on Information Processing. 2008. p. 198.Doukas, C., & Maglogiannis, I. (2010). A Fast Mobile Face Recognition System for Android OS Based on Eigenfaces Decomposition. Artificial Intelligence Applications and Innovations, 295-302. doi:10.1007/978-3-642-16239-8_39Huang P, Huang Y, Wang W, Wang L. Deep embedding network for clustering. Proceedings—International Conference on Pattern Recognition. 2014. pp. 1532–1537. doi: https://doi.org/10.1109/ICPR.2014.272Dizaji KG, Herandi A, Deng C, Cai W, Huang H. Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization. Proceedings of the IEEE International Conference on Computer Vision. 2017. doi: https://doi.org/10.1109/ICCV.2017.612Xie J, Girshick R, Farhadi A. Unsupervised deep embedding for clustering analysis [Internet]. Proceedings of the 33rd International Conference on International Conference on Machine Learning—Volume 48. JMLR.org; 2016. pp. 478–487. Available: https://dl.acm.org/citation.cfm?id=3045442Nousi, P., & Tefas, A. (2017). Discriminatively Trained Autoencoders for Fast and Accurate Face Recognition. Communications in Computer and Information Science, 205-215. doi:10.1007/978-3-319-65172-9_18Sirovich, L., & Kirby, M. (1987). Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, 4(3), 519. doi:10.1364/josaa.4.00051

Multi-View Face Recognition From Single RGBD Models of the Faces

This work takes important steps towards solving the following problem of current interest: Assuming that each individual in a population can be modeled by a single frontal RGBD face image, is it possible to carry out face recognition for such a population using multiple 2D images captured from arbitrary viewpoints? Although the general problem as stated above is extremely challenging, it encompasses subproblems that can be addressed today. The subproblems addressed in this work relate to: (1) Generating a large set of viewpoint dependent face images from a single RGBD frontal image for each individual; (2) using hierarchical approaches based on view-partitioned subspaces to represent the training data; and (3) based on these hierarchical approaches, using a weighted voting algorithm to integrate the evidence collected from multiple images of the same face as recorded from different viewpoints. We evaluate our methods on three datasets: a dataset of 10 people that we created and two publicly available datasets which include a total of 48 people. In addition to providing important insights into the nature of this problem, our results show that we are able to successfully recognize faces with accuracies of 95% or higher, outperforming existing state-of-the-art face recognition approaches based on deep convolutional neural networks

Infrared face recognition: a comprehensive review of methodologies and databases

Automatic face recognition is an area with immense practical potential which includes a wide range of commercial and law enforcement applications. Hence it is unsurprising that it continues to be one of the most active research areas of computer vision. Even after over three decades of intense research, the state-of-the-art in face recognition continues to improve, benefitting from advances in a range of different research fields such as image processing, pattern recognition, computer graphics, and physiology. Systems based on visible spectrum images, the most researched face recognition modality, have reached a significant level of maturity with some practical success. However, they continue to face challenges in the presence of illumination, pose and expression changes, as well as facial disguises, all of which can significantly decrease recognition accuracy. Amongst various approaches which have been proposed in an attempt to overcome these limitations, the use of infrared (IR) imaging has emerged as a particularly promising research direction. This paper presents a comprehensive and timely review of the literature on this subject. Our key contributions are: (i) a summary of the inherent properties of infrared imaging which makes this modality promising in the context of face recognition, (ii) a systematic review of the most influential approaches, with a focus on emerging common trends as well as key differences between alternative methodologies, (iii) a description of the main databases of infrared facial images available to the researcher, and lastly (iv) a discussion of the most promising avenues for future research.Comment: Pattern Recognition, 2014. arXiv admin note: substantial text overlap with arXiv:1306.160