Automatic classification of human facial features based on their appearance

[EN] Classification or typology systems used to categorize different human body parts have existed for many years. Nevertheless, there are very few taxonomies of facial features. Ergonomics, forensic anthropology, crime prevention or new human-machine interaction systems and online activities, like e-commerce, e-learning, games, dating or social networks, are fields in which classifications of facial features are useful, for example, to create digital interlocutors that optimize the interactions between human and machines. However, classifying isolated facial features is difficult for human observers. Previous works reported low inter-observer and intra-observer agreement in the evaluation of facial features. This work presents a computer-based procedure to automatically classify facial features based on their global appearance. This procedure deals with the difficulties associated with classifying features using judgements from human observers, and facilitates the development of taxonomies of facial features. Taxonomies obtained through this procedure are presented for eyes, mouths and noses.Fuentes-Hurtado, F.; Diego-Mas, JA.; Naranjo Ornedo, V.; Alcañiz Raya, ML. (2019). Automatic classification of human facial features based on their appearance. PLoS ONE. 14(1):1-20. https://doi.org/10.1371/journal.pone.0211314S120141Damasio, A. R. (1985). Prosopagnosia. Trends in Neurosciences, 8, 132-135. doi:10.1016/0166-2236(85)90051-7Bruce, V., & Young, A. (1986). Understanding face recognition. British Journal of Psychology, 77(3), 305-327. doi:10.1111/j.2044-8295.1986.tb02199.xTodorov, A. (2011). Evaluating Faces on Social Dimensions. Social Neuroscience, 54-76. doi:10.1093/acprof:oso/9780195316872.003.0004Little, A. C., Burriss, R. P., Jones, B. C., & Roberts, S. C. (2007). Facial appearance affects voting decisions. Evolution and Human Behavior, 28(1), 18-27. doi:10.1016/j.evolhumbehav.2006.09.002Porter, J. P., & Olson, K. L. (2001). Anthropometric Facial Analysis of the African American Woman. Archives of Facial Plastic Surgery, 3(3), 191-197. doi:10.1001/archfaci.3.3.191Gündüz Arslan, S., Genç, C., Odabaş, B., & Devecioğlu Kama, J. (2007). Comparison of Facial Proportions and Anthropometric Norms Among Turkish Young Adults With Different Face Types. Aesthetic Plastic Surgery, 32(2), 234-242. doi:10.1007/s00266-007-9049-yFerring, V., & Pancherz, H. (2008). Divine proportions in the growing face. American Journal of Orthodontics and Dentofacial Orthopedics, 134(4), 472-479. doi:10.1016/j.ajodo.2007.03.027Mane, D. R., Kale, A. D., Bhai, M. B., & Hallikerimath, S. (2010). Anthropometric and anthroposcopic analysis of different shapes of faces in group of Indian population: A pilot study. Journal of Forensic and Legal Medicine, 17(8), 421-425. doi:10.1016/j.jflm.2010.09.001Ritz-Timme, S., Gabriel, P., Tutkuviene, J., Poppa, P., Obertová, Z., Gibelli, D., … Cattaneo, C. (2011). Metric and morphological assessment of facial features: A study on three European populations. Forensic Science International, 207(1-3), 239.e1-239.e8. doi:10.1016/j.forsciint.2011.01.035Ritz-Timme, S., Gabriel, P., Obertovà, Z., Boguslawski, M., Mayer, F., Drabik, A., … Cattaneo, C. (2010). A new atlas for the evaluation of facial features: advantages, limits, and applicability. International Journal of Legal Medicine, 125(2), 301-306. doi:10.1007/s00414-010-0446-4Kong, S. G., Heo, J., Abidi, B. R., Paik, J., & Abidi, M. A. (2005). Recent advances in visual and infrared face recognition—a review. Computer Vision and Image Understanding, 97(1), 103-135. doi:10.1016/j.cviu.2004.04.001Tavares, G., Mourão, A., & Magalhães, J. (2016). Crowdsourcing facial expressions for affective-interaction. Computer Vision and Image Understanding, 147, 102-113. doi:10.1016/j.cviu.2016.02.001Buckingham, G., DeBruine, L. M., Little, A. C., Welling, L. L. M., Conway, C. A., Tiddeman, B. P., & Jones, B. C. (2006). Visual adaptation to masculine and feminine faces influences generalized preferences and perceptions of trustworthiness. Evolution and Human Behavior, 27(5), 381-389. doi:10.1016/j.evolhumbehav.2006.03.001Boberg M, Piippo P, Ollila E. Designing Avatars. DIMEA ‘08 Proc 3rd Int Conf Digit Interact Media Entertain Arts. ACM; 2008; 232–239. doi: https://doi.org/10.1145/1413634.1413679Rojas Q., M., Masip, D., Todorov, A., & Vitria, J. (2011). Automatic Prediction of Facial Trait Judgments: Appearance vs. Structural Models. PLoS ONE, 6(8), e23323. doi:10.1371/journal.pone.0023323Laurentini, A., & Bottino, A. (2014). Computer analysis of face beauty: A survey. Computer Vision and Image Understanding, 125, 184-199. doi:10.1016/j.cviu.2014.04.006Alemany S, Gonzalez J, Nacher B, Soriano C, Arnaiz C, Heras H. Anthropometric survey of the Spanish female population aimed at the apparel industry. Proceedings of the 2010 Intl Conference on 3D Body scanning Technologies. 2010. pp. 307–315.Vinué, G., Epifanio, I., & Alemany, S. (2015). Archetypoids: A new approach to define representative archetypal data. Computational Statistics & Data Analysis, 87, 102-115. doi:10.1016/j.csda.2015.01.018Jee, S., & Yun, M. H. (2016). An anthropometric survey of Korean hand and hand shape types. International Journal of Industrial Ergonomics, 53, 10-18. doi:10.1016/j.ergon.2015.10.004Kim, N.-S., & Do, W.-H. (2014). Classification of Elderly Women’s Foot Type. Journal of the Korean Society of Clothing and Textiles, 38(3), 305-320. doi:10.5850/jksct.2014.38.3.305Sarakon P, Charoenpong T, Charoensiriwath S. Face shape classification from 3D human data by using SVM. The 7th 2014 Biomedical Engineering International Conference. IEEE; 2014. pp. 1–5. doi: https://doi.org/10.1109/BMEiCON.2014.7017382PRESTON, T. A., & SINGH, M. (1972). Redintegrated Somatotyping. Ergonomics, 15(6), 693-700. doi:10.1080/00140137208924469Lin, Y.-L., & Lee, K.-L. (1999). Investigation of anthropometry basis grouping technique for subject classification. Ergonomics, 42(10), 1311-1316. doi:10.1080/001401399184965Malousaris, G. G., Bergeles, N. K., Barzouka, K. G., Bayios, I. A., Nassis, G. P., & Koskolou, M. D. (2008). Somatotype, size and body composition of competitive female volleyball players. Journal of Science and Medicine in Sport, 11(3), 337-344. doi:10.1016/j.jsams.2006.11.008Carvalho, P. V. R., dos Santos, I. L., Gomes, J. O., Borges, M. R. S., & Guerlain, S. (2008). Human factors approach for evaluation and redesign of human–system interfaces of a nuclear power plant simulator. Displays, 29(3), 273-284. doi:10.1016/j.displa.2007.08.010Fabri M, Moore D. The use of emotionally expressive avatars in Collaborative Virtual Environments. AISB’05 Convention:Proceedings of the Joint Symposium on Virtual Social Agents: Social Presence Cues for Virtual Humanoids Empathic Interaction with Synthetic Characters Mind Minding Agents. 2005. pp. 88–94. doi:citeulike-article-id:790934Sukhija, P., Behal, S., & Singh, P. (2016). Face Recognition System Using Genetic Algorithm. Procedia Computer Science, 85, 410-417. doi:10.1016/j.procs.2016.05.183Trescak T, Bogdanovych A, Simoff S, Rodriguez I. Generating diverse ethnic groups with genetic algorithms. Proceedings of the 18th ACM symposium on Virtual reality software and technology—VRST ‘12. New York, New York, USA: ACM Press; 2012. p. 1. doi: https://doi.org/10.1145/2407336.2407338Vanezis, P., Lu, D., Cockburn, J., Gonzalez, A., McCombe, G., Trujillo, O., & Vanezis, M. (1996). Morphological Classification of Facial Features in Adult Caucasian Males Based on an Assessment of Photographs of 50 Subjects. Journal of Forensic Sciences, 41(5), 13998J. doi:10.1520/jfs13998jTamir, A. (2011). Numerical Survey of the Different Shapes of the Human Nose. Journal of Craniofacial Surgery, 22(3), 1104-1107. doi:10.1097/scs.0b013e3182108eb3Tamir, A. (2013). Numerical Survey of the Different Shapes of Human Chin. Journal of Craniofacial Surgery, 24(5), 1657-1659. doi:10.1097/scs.0b013e3182942b77Richler, J. J., Cheung, O. S., & Gauthier, I. (2011). Holistic Processing Predicts Face Recognition. Psychological Science, 22(4), 464-471. doi:10.1177/0956797611401753Taubert, J., Apthorp, D., Aagten-Murphy, D., & Alais, D. (2011). The role of holistic processing in face perception: Evidence from the face inversion effect. Vision Research, 51(11), 1273-1278. doi:10.1016/j.visres.2011.04.002Donnelly, N., & Davidoff, J. (1999). The Mental Representations of Faces and Houses: Issues Concerning Parts and Wholes. Visual Cognition, 6(3-4), 319-343. doi:10.1080/135062899395000Davidoff, J., & Donnelly, N. (1990). Object superiority: A comparison of complete and part probes. Acta Psychologica, 73(3), 225-243. doi:10.1016/0001-6918(90)90024-aTanaka, J. W., & Farah, M. J. (1993). Parts and Wholes in Face Recognition. The Quarterly Journal of Experimental Psychology Section A, 46(2), 225-245. doi:10.1080/14640749308401045Wang, R., Li, J., Fang, H., Tian, M., & Liu, J. (2012). Individual Differences in Holistic Processing Predict Face Recognition Ability. Psychological Science, 23(2), 169-177. doi:10.1177/0956797611420575Rhodes, G., Ewing, L., Hayward, W. G., Maurer, D., Mondloch, C. J., & Tanaka, J. W. (2009). Contact and other-race effects in configural and component processing of faces. British Journal of Psychology, 100(4), 717-728. doi:10.1348/000712608x396503Miller, G. A. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101(2), 343-352. doi:10.1037/0033-295x.101.2.343Scharff, A., Palmer, J., & Moore, C. M. (2011). Evidence of fixed capacity in visual object categorization. Psychonomic Bulletin & Review, 18(4), 713-721. doi:10.3758/s13423-011-0101-1Meyers, E., & Wolf, L. (2007). Using Biologically Inspired Features for Face Processing. International Journal of Computer Vision, 76(1), 93-104. doi:10.1007/s11263-007-0058-8Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681-685. doi:10.1109/34.927467Ahonen, T., Hadid, A., & Pietikainen, M. (2006). Face Description with Local Binary Patterns: Application to Face Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 2037-2041. doi:10.1109/tpami.2006.244Belhumeur, P. N., Hespanha, J. P., & Kriegman, D. J. (1997). Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 711-720. doi:10.1109/34.598228Turk, M., & Pentland, A. (1991). Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 3(1), 71-86. doi:10.1162/jocn.1991.3.1.71Klare B, Jain AK. On a taxonomy of facial features. IEEE 4th International Conference on Biometrics: Theory, Applications and Systems, BTAS 2010. IEEE; 2010. pp. 1–8. doi: https://doi.org/10.1109/BTAS.2010.5634533Chihaoui, M., Elkefi, A., Bellil, W., & Ben Amar, C. (2016). A Survey of 2D Face Recognition Techniques. Computers, 5(4), 21. doi:10.3390/computers5040021Ma, D. S., Correll, J., & Wittenbrink, B. (2015). The Chicago face database: A free stimulus set of faces and norming data. Behavior Research Methods, 47(4), 1122-1135. doi:10.3758/s13428-014-0532-5Asthana A, Zafeiriou S, Cheng S, Pantic M. Incremental face alignment in the wild. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2014. pp. 1859–1866. doi: https://doi.org/10.1109/CVPR.2014.240Bag S, Barik S, Sen P, Sanyal G. A statistical nonparametric approach of face recognition: combination of eigenface & modified k-means clustering. Proceedings Second International Conference on Information Processing. 2008. p. 198.Doukas, C., & Maglogiannis, I. (2010). A Fast Mobile Face Recognition System for Android OS Based on Eigenfaces Decomposition. Artificial Intelligence Applications and Innovations, 295-302. doi:10.1007/978-3-642-16239-8_39Huang P, Huang Y, Wang W, Wang L. Deep embedding network for clustering. Proceedings—International Conference on Pattern Recognition. 2014. pp. 1532–1537. doi: https://doi.org/10.1109/ICPR.2014.272Dizaji KG, Herandi A, Deng C, Cai W, Huang H. Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization. Proceedings of the IEEE International Conference on Computer Vision. 2017. doi: https://doi.org/10.1109/ICCV.2017.612Xie J, Girshick R, Farhadi A. Unsupervised deep embedding for clustering analysis [Internet]. Proceedings of the 33rd International Conference on International Conference on Machine Learning—Volume 48. JMLR.org; 2016. pp. 478–487. Available: https://dl.acm.org/citation.cfm?id=3045442Nousi, P., & Tefas, A. (2017). Discriminatively Trained Autoencoders for Fast and Accurate Face Recognition. Communications in Computer and Information Science, 205-215. doi:10.1007/978-3-319-65172-9_18Sirovich, L., & Kirby, M. (1987). Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A, 4(3), 519. doi:10.1364/josaa.4.00051

Predictive biometrics: A review and analysis of predicting personal characteristics from biometric data

Interest in the exploitation of soft biometrics information has continued to develop over the last decade or so. In comparison with traditional biometrics, which focuses principally on person identification, the idea of soft biometrics processing is to study the utilisation of more general information regarding a system user, which is not necessarily unique. There are increasing indications that this type of data will have great value in providing complementary information for user authentication. However, the authors have also seen a growing interest in broadening the predictive capabilities of biometric data, encompassing both easily definable characteristics such as subject age and, most recently, `higher level' characteristics such as emotional or mental states. This study will present a selective review of the predictive capabilities, in the widest sense, of biometric data processing, providing an analysis of the key issues still adequately to be addressed if this concept of predictive biometrics is to be fully exploited in the future

STV-based Video Feature Processing for Action Recognition

In comparison to still image-based processes, video features can provide rich and intuitive information about dynamic events occurred over a period of time, such as human actions, crowd behaviours, and other subject pattern changes. Although substantial progresses have been made in the last decade on image processing and seen its successful applications in face matching and object recognition, video-based event detection still remains one of the most difficult challenges in computer vision research due to its complex continuous or discrete input signals, arbitrary dynamic feature definitions, and the often ambiguous analytical methods. In this paper, a Spatio-Temporal Volume (STV) and region intersection (RI) based 3D shape-matching method has been proposed to facilitate the definition and recognition of human actions recorded in videos. The distinctive characteristics and the performance gain of the devised approach stemmed from a coefficient factor-boosted 3D region intersection and matching mechanism developed in this research. This paper also reported the investigation into techniques for efficient STV data filtering to reduce the amount of voxels (volumetric-pixels) that need to be processed in each operational cycle in the implemented system. The encouraging features and improvements on the operational performance registered in the experiments have been discussed at the end

Selecting surface features for accurate multi-camera surface reconstruction

This paper proposes a novel feature detector for selecting local textures that are suitable for accurate multi-camera surface reconstruction, and in particular planar patch fitting techniques. This approach is in contrast to conventional feature detectors, which focus on repeatability under scale and affine transformations rather than suitability for multi-camera reconstruction techniques. The proposed detector selects local textures that are sensitive to affine transformations, which is a fundamental requirement for accurate patch fitting. The proposed detector is evaluated against the SIFT detector on a synthetic dataset and the fitted patches are compared against ground truth. The experiments show that patches originating from the proposed detector are fitted more accurately to the visible surfaces than those originating from SIFT keypoints. In addition, the detector is evaluated on a performance capture studio dataset to show the real-world application of the proposed detector