Automatic Kinship Verification in Unconstrained Faces using Deep Learning

Kinship verification has a number of applications such as organizing large collections of images and recognizing resemblances among humans. Identifying kinship relations has also garnered interest due to several potential applications in security and surveillance and organizing and tagging the enormous number of videos being uploaded on the Internet. This dissertation has a five-fold contribution where first, a study is conducted to gain insight into the kinship verification process used by humans. Besides this, two separate deep learning based methods are proposed to solve kinship verification in images and videos. Other contributions of this research include interlinking face verification with kinship verification and creation of two kinship databases to facilitate research in this field. WVU Kinship Database is created which consists of multiple images per subject to facilitate kinship verification research. Next, kinship video (KIVI) database of more than 500 individuals with variations due to illumination, pose, occlusion, ethnicity, and expression is collected for this research. It comprises a total of 355 true kin video pairs with over 250,000 still frames. In this dissertation, a human study is conducted to understand the capabilities of human mind and to identify the discriminatory areas of a face that facilitate kinship-cues. The visual stimuli presented to the participants determines their ability to recognize kin relationship using the whole face as well as specific facial regions. The effect of participant gender, age, and kin-relation pair of the stimulus is analyzed using quantitative measures such as accuracy, discriminability index d′, and perceptual information entropy. Next, utilizing the information obtained from the human study, a hierarchical Kinship Verification via Representation Learning (KVRL) framework is utilized to learn the representation of different face regions in an unsupervised manner. We propose a novel approach for feature representation termed as filtered contractive deep belief networks (fcDBN). The proposed feature representation encodes relational information present in images using filters and contractive regularization penalty. A compact representation of facial images of kin is extracted as the output from the learned model and a multi-layer neural network is utilized to verify the kin accurately. The results show that the proposed deep learning framework (KVRL-fcDBN) yields state-of-the-art kinship verification accuracy on the WVU Kinship database and on four existing benchmark datasets. Additionally, we propose a new deep learning framework for kinship verification in unconstrained videos using a novel Supervised Mixed Norm regularization Autoencoder (SMNAE). This new autoencoder formulation introduces class-specific sparsity in the weight matrix. The proposed three-stage SMNAE based kinship verification framework utilizes the learned spatio-temporal representation in the video frames for verifying kinship in a pair of videos. The effectiveness of the proposed framework is demonstrated on the KIVI database and six existing kinship databases. On the KIVI database, SMNAE yields videobased kinship verification accuracy of 83.18% which is at least 3.2% better than existing algorithms. The algorithm is also evaluated on six publicly available kinship databases and compared with best reported results. It is observed that the proposed SMNAE consistently yields best results on all the databases. Finally, we end by discussing the connections between face verification and kinship verification research. We explore the area of self-kinship which is age-invariant face recognition. Further, kinship information is used as a soft biometric modality to boost the performance of face verification via product of likelihood ratio and support vector machine based approaches. Using the proposed KVRL-fcDBN framework, an improvement of over 20% is observed in the performance of face verification. By addressing several problems of limited samples per kinship dataset, introducing real-world variations in unconstrained databases and designing two deep learning frameworks, this dissertation improves the understanding of kinship verification across humans and the performance of automated systems. The algorithms proposed in this research have been shown to outperform existing algorithms across six different kinship databases and has till date the best reported results in this field

Deep Ear Biometrics for Gender Classification

Human gender classification based on biometric features is a major concern for computer vision due to its vast variety of applications. The human ear is popular among researchers as a soft biometric trait, because it is less affected by age or changing circumstances, and is non-intrusive. In this study, we have developed a deep convolutional neural network (CNN) model for automatic gender classification using the samples of ear images. The performance is evaluated using four cutting-edge pre-trained CNN models. In terms of trainable parameters, the proposed technique requires significantly less computational complexity. The proposed model has achieved 93% accuracy on the EarVN1.0 ear dataset.Comment: 10 pages, 4 figures, 2 table

Fusion features ensembling models using Siamese convolutional neural network for kinship verification

Family is one of the most important entities in the community. Mining the genetic information through facial images is increasingly being utilized in wide range of real-world applications to facilitate family members tracing and kinship analysis to become remarkably easy, inexpensive, and fast as compared to the procedure of profiling Deoxyribonucleic acid (DNA). However, the opportunities of building reliable models for kinship recognition are still suffering from the insufficient determination of the familial features, unstable reference cues of kinship, and the genetic influence factors of family features. This research proposes enhanced methods for extracting and selecting the effective familial features that could provide evidences of kinship leading to improve the kinship verification accuracy through visual facial images. First, the Convolutional Neural Network based on Optimized Local Raw Pixels Similarity Representation (OLRPSR) method is developed to improve the accuracy performance by generating a new matrix representation in order to remove irrelevant information. Second, the Siamese Convolutional Neural Network and Fusion of the Best Overlapping Blocks (SCNN-FBOB) is proposed to track and identify the most informative kinship clues features in order to achieve higher accuracy. Third, the Siamese Convolutional Neural Network and Ensembling Models Based on Selecting Best Combination (SCNN-EMSBC) is introduced to overcome the weak performance of the individual image and classifier. To evaluate the performance of the proposed methods, series of experiments are conducted using two popular benchmarking kinship databases; the KinFaceW-I and KinFaceW-II which then are benchmarked against the state-of-art algorithms found in the literature. It is indicated that SCNN-EMSBC method achieves promising results with the average accuracy of 92.42% and 94.80% on KinFaceW-I and KinFaceW-II, respectively. These results significantly improve the kinship verification performance and has outperformed the state-of-art algorithms for visual image-based kinship verification

Facial kinship verification with large age variation using deep linear metric learning

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Mecânica, 2018.Aparência facial afeta como os seres humanos interagem. É como os parentes são visualmente identificados para determinar como as interações sociais ocorrem. Humanos podem identificar relações de parentesco baseado-se apenas no rosto. Intrinsecamente, dar a capacidade de detectar relações de parentesco a computadores pode aumentar a utilidade dos mesmos em nossas sociedade. Esta pesquisa propôs uma metodo para o problema de verificação de parentesco com uma nova abordagem não-contextual usando um conjunto de dados com grande variação de idade, aplicando nosso método proposto de Aprendizagem Métrica Linear Profunda (DLML). Nosso método aproveita várias arquiteturas de aprendizado profundo treinadas com conjuntos de dados faciais massivos. O conhecimento adquirido em tarefas tradicionais de reconhecimento facial é adaptado para alimentar um modelo linear de aprendizagem métrica. O método proposto foi capaz de alcançar um desempenho melhor do que outros métodos de verificação de parentesco com contexto em testes que são mais difíceis do que os usados em métodos anteriores com o conjunto de dados do banco de dados do UB Kinface. Os resultados mostram que nosso método pode usar o conhecimento de arquiteturas de aprendizagem profunda treinadas para realizar tarefas de reconhecimento facial com grandes conjuntos de dados para resolver a verificação de parentesco no banco de dados UB Kinface com robustez a grandes diferenças de idade presentes no conjunto de dados. Nosso método também oferece aplicabilidade aprimorada quando comparado a métodos anteriores em situações do mundo real, porque elimina a necessidade de saber/detectar e tratar grandes variações de idade para realizar a verificação de parentesco.Facial appearance affects how humans interact. It is how relatives are visually identified to determine how social interactions proceed. Humans can identify kin relations based only on the face. Intrinsically, giving the ability to detect kin relations to computers can improve their usefulness in our daily lives. This research proposed a solution to the kinship verification problem with a novel non-context-aware approach using a dataset with large age variation by applying our proposed method Deep Linear Metric Learning(DLML). Our method leverages multiple deep learning architectures trained with massive facial datasets. The knowledge acquired on traditional facial recognition tasks is re-purposed to feed a linear metric learning model. The proposed method was able to achieve better performance than other context-aware methods on tests that are inherently more difficult than the ones used on previous methods with the UB Kinface dataset. The results show that our method can use the knowledge of deep learning architectures trained to perform mainstream facial recognition tasks with massive datasets to solve kinship verification on the UB Kinface database with robustness towards large age differences present on the dataset. Our method also offers enhanced applicability when compared to previous methods on real-world situations, because it removes the necessity of knowing/detecting and treating large age variations to perform kinship verification

The Role of Educational Technology in Caregiving

Huge demographic and socio-economic changes are part of the experience of present societies. One consequence is the aging of the population and increasingly more people without the capacity for self-care. The provision of intergenerational care, namely caring for the older individuals, is a focus of attention for health professionals, but is also part of the political and social agenda. There is a need to regulate, support, and facilitate the daily life of families who have a dependent aged member. In contemporary societies, the use of information and communication technologies (ICT) is an important driver of innovation, responsible for a large transformation of living standards and new social behaviors. Within the scope of the provision of informal care, ICT can provide a great support, representing a primordial tool for updating the organizations in order to improve their efficiency, incorporating and making available services, and anticipating needs. Thus, the development of equipment, electronic applications, and websites for the elderly or their family caregivers should be conceptualized and customized to the profile of these users. A major challenge faced by healthcare institutions is to focus their services by organizing them around citizens’ needs