Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review

Facial emotional processing is one of the most important activities in effective calculations, engagement with people and computers, machine vision, video game testing, and consumer research. Facial expressions are a form of nonverbal communication, as they reveal a person's inner feelings and emotions. Extensive attention to Facial Expression Recognition (FER) has recently been received as facial expressions are considered. As the fastest communication medium of any kind of information. Facial expression recognition gives a better understanding of a person's thoughts or views and analyzes them with the currently trending deep learning methods. Accuracy rate sharply compared to traditional state-of-the-art systems. This article provides a brief overview of the different FER fields of application and publicly accessible databases used in FER and studies the latest and current reviews in FER using Convolution Neural Network (CNN) algorithms. Finally, it is observed that everyone reached good results, especially in terms of accuracy, with different rates, and using different data sets, which impacts the results

Binary Pattern Analysis for 3D Facial Action Unit Detection

In this paper we propose new binary pattern features for use in the problem of 3D facial action unit (AU) detection. Two representations of 3D facial geometries are employed, the depth map and the Azimuthal Projection Distance Image (APDI). To these the traditional Local Binary Pattern is applied, along with Local Phase Quantisation, Gabor filters and Monogenic filters, followed by the binary pattern feature extraction method. Feature vectors are formed for each feature type through concatenation of histograms formed from the resulting binary numbers. Feature selection is then performed using a two-stage GentleBoost approach. Finally, we apply Support Vector Machines as classifiers for detection of each AU. This system is tested in two ways. First we perform 10-fold cross-validation on the Bosphorus database, and then we perform cross-database testing by training on this database and then testing on apex frames from the D3DFACS database, achieving promising results in both

Towards spatial and temporal analysis of facial expressions in 3D data

Facial expressions are one of the most important means for communication of emotions and meaning. They are used to clarify and give emphasis, to express intentions, and form a crucial part of any human interaction. The ability to automatically recognise and analyse expressions could therefore prove to be vital in human behaviour understanding, which has applications in a number of areas such as psychology, medicine and security. 3D and 4D (3D+time) facial expression analysis is an expanding field, providing the ability to deal with problems inherent to 2D images, such as out-of-plane motion, head pose, and lighting and illumination issues. Analysis of data of this kind requires extending successful approaches applied to the 2D problem, as well as the development of new techniques. The introduction of recent new databases containing appropriate expression data, recorded in 3D or 4D, has allowed research into this exciting area for the first time. This thesis develops a number of techniques, both in 2D and 3D, that build towards a complete system for analysis of 4D expressions. Suitable feature types, designed by employing binary pattern methods, are developed for analysis of 3D facial geometry data. The full dynamics of 4D expressions are modelled, through a system reliant on motion-based features, to demonstrate how the different components of the expression (neutral-onset-apex-offset) can be distinguished and harnessed. Further, the spatial structure of expressions is harnessed to improve expression component intensity estimation in 2D videos. Finally, it is discussed how this latter step could be extended to 3D facial expression analysis, and also combined with temporal analysis. Thus, it is demonstrated that both spatial and temporal information, when combined with appropriate 3D features, is critical in analysis of 4D expression data.Open Acces

High Performance Video Stream Analytics System for Object Detection and Classification

Due to the recent advances in cameras, cell phones and camcorders, particularly the resolution at which they can record an image/video, large amounts of data are generated daily. This video data is often so large that manually inspecting it for object detection and classification can be time consuming and error prone, thereby it requires automated analysis to extract useful information and meta-data. The automated analysis from video streams also comes with numerous challenges such as blur content and variation in illumination conditions and poses. We investigate an automated video analytics system in this thesis which takes into account the characteristics from both shallow and deep learning domains. We propose fusion of features from spatial frequency domain to perform highly accurate blur and illumination invariant object classification using deep learning networks. We also propose the tuning of hyper-parameters associated with the deep learning network through a mathematical model. The mathematical model used to support hyper-parameter tuning improved the performance of the proposed system during training. The outcomes of various hyper-parameters on system's performance are compared. The parameters that contribute towards the most optimal performance are selected for the video object classification. The proposed video analytics system has been demonstrated to process a large number of video streams and the underlying infrastructure is able to scale based on the number and size of the video stream(s) being processed. The extensive experimentation on publicly available image and video datasets reveal that the proposed system is significantly more accurate and scalable and can be used as a general purpose video analytics system.N/

Facial recognition in hexagonal domain - a frontier approach

DATA AVAILABILITY : Data sharing does not apply to this article, as no new datasets were generated or analyzed during the current study.Many face-recognition (FR) methods have been proposed thus far. Although FR has achieved wisdom in square pixel-based image processing (SIP) due to many studies, this wisdom has not been transferred to Hexagonal pixel-based image processing (HIP) until now. This study presents HIP versions of the most basic texture extraction studies in SIP, namely Gray-Level-Co-occurrence-Matrices (GLCM), Local Binary Pattern (LBP), and our recent work, local-holistic graph-based descriptor (LHGPD). The images are first transformed from the SIP domain to the HIP domain. The HIP domain equivalences (HexGLCM, HexLBP, and HexLHGPD) of the SIP domain GLCM, LBP, and LHGPD are then established. Finally, the facial recognition performances of the SIP and HIP domain versions of GLCM, LBP, and LHGPD are evaluated and compared on the primary data sets. The results of the experiments reveal that HIP domain GLCM, LBP, and LHGPD show a par performance, surpassing them in places when compared to their counterparts in the SIP domain regarding face recognition accuracy.https://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=6287639am2024Electrical, Electronic and Computer EngineeringSDG-09: Industry, innovation and infrastructur

Dissecting the genetic basis of neurodevelopmental disorders and demyelinating neuropathies

The understanding of the pathophysiology of most rare, complex neurological disorders has been elusive, especially in the case of complex demyelinating neuropathies and neurodevelopmental disorders. In my work, I learnt to employ two main techniques that will help advance the search for better understanding of neurodevelopmental disorders: next generation sequencing and functional validation of rare genetic variants. The main aim of my research was to establish the genetic diagnosis in several patients affected by complex syndromes such as peripheral neuropathy with central nervous system involvement (Chapter 3), neurodevelopmental disorders (Chapter 4) and epilepsy (Chapter 5). The phenotypic and genotypic correlations of identified gene variants were investigated in these chapters and is a profound theme in my project. To achieve this, an integrated approach combining next generation sequencing (NGS) technology, homozygosity mapping, array genotyping, traditional Sanger sequencing and functional experiments was undertaken. Firstly, I describe the work performed in an attempt to identify the causative gene in a cohort of young children presented with an early-onset hereditary form of chronic inflammatory demyelinating polyneuropathy with a central and peripheral involvement. My key findings were that: i) neurofascin is the first gene causally responsible for an inherited disorder that resembles CIDP, ii) this is the largest clinical cohort to date of patients with NFASC mutations with 10 individuals, and iii) the functional evidence implicate the major protein isoforms, which were also shown to be the main targets for the autoantibodies in CIDP pathogenesis. Secondly, I describe the work done on various neurodevelopmental disorder (NDD) genes, with particular focus on a newly identified gene presenting with a complex neurodevelopmental phenotype comprised of developmental delay, epilepsy, and/or a demyelinating neuropathy. My key findings were that: i) NARS1, a cytoplasmic aminoacyl-tRNA synthetase enzyme can be causative for this disorder by either a de-novo heterozygous or a biallelic inheritance mode, ii) functional investigations showed reduced aminoacylation activity in the disease-associated biallelic mutations using fibroblasts and iNPCs transcriptomics, suggesting that the majority of NARS1 mutations cause a loss of function of the protein by reduced expression and disruption of dimer formation suggesting a loss-of-function mechanism, and iii) increased yeast growth in the disease-associated heterozygous mutations showing near normal protein expression are suggestive of a gain-of-function mechanism. Finally, I describe the work done on two relative new genes (PIGS and TARS1) in an attempt to expand the patient phenotypic spectrum, as well as an interesting candidate gene (SLITRK3) linked with epilepsy. I present my understanding for disease-gene discovery that will enable me and other members of the neurogenetics field to identify disease-mechanisms and address important gaps of translational research into rare neurological diseases such as those described in this thesis

Characterisation of two retinal proteins, peripherin/RDs and retinaldehyde dehydrogenase type 2

Peripherin/rds (PRDS) is an integral membrane protein that is found in the outer segment of photoreceptor cells, in which dysfunctions cause a number of retinal degenerations including retinitis pigmentosa and macular degeneration. There are over 60 disease causing mutations associated with this gene, most of which are found in the lumen (L3-4) loop domain of the protein. Retinaldehyde dehydrogenase type 2 (Raldh-2) is an enzyme found in retinal pigment epithelial layer (RPE) which is required for generation of retinoic acid (RA) from retinaldehyde (RAL). RA is required for the development of many embryonic structures including the eye, heart and spinal cord and in the regulation of many genes that are involved in the development of these structures. The purpose of this study was to characterize the L3-4 loop region of PRDS through the recombinant expression of the L3-4 loop as a fusion protein with the maltose binding protein from E. coli. Moreover, some mutations known to cause retinal degenerations, R172W/G (macular degeneration and pattern dystrophy), L185P (digenic retinitis pigmentosa) and C165Y (retinitis pigmentosa) were introduced into PRDS by site directed mutagenesis. The circular dichroism spectra of wild type and mutant loop were compared, and the results were consistent with these mutations having an effect on the topology of the loop region. The aim of the expression of the Raldh-2 enzyme was to obtain a detailed study of its mechanism. The enzyme was overexpressed as a His- tagged recombinant protein in E. coli. The Raldh-2 protein sequence was modelled onto a known aldehyde dehydrogenase structure, the following conserved amino acid residues were found to be located in the active site, C301-2, S107, N169 and E399. These were all mutated to alanine and kinetic characterization of the mutant enzymes revealed that these amino acid residues are important for catalysis

Bioinformatics tools for the genetic dissection of complex traits in chickens

This thesis explores the genetic characterization of the mechanisms underlying complex traits in chicken through the use and development of bioinformatics tools. The characterization of quantitative trait loci controlling complex traits has proven to be very challenging. This thesis comprises the study of experimental designs, annotation procedures and functional analyses. These represent some of the main ‘bottlenecks’ involved in the integration of QTLs with the biological interpretation of high-throughput technologies. The thesis begins with an investigation of the bioinformatics tools and procedures available for genome research, briefly reviewing microarray technology and commonly applied experimental designs. A targeted experimental design based on the concept of genetical genomics is then presented and applied in order to study a known functional QTL responsible for chicken body weight. This approach contrasts the gene expression levels of two alternative QTL genotypes, hence narrowing the QTL-phenotype gap, and, giving a direct quantification of the link between the genotypes and the genetic responses. Potential candidate genes responsible for the chicken body weight QTL are identified by using the location of the genes, their expression and biological significance. In order to deal with the multiple sources of information and exploit the data effectively, a systematic approach and a relational database were developed to improve the annotation of the probes of the ARK-Genomics G. gallus 13K v4.0 cDNA array utilized on the experiment. To follow up the investigation of the targeted genetical genomics study, a detailed functional analysis is performed on the dataset. The aim is to identify the downstream effects through the identification of functional variation found in pathways, and secondly to achieve a further characterization of potential candidate genes by using comparative genomics and sequence analyses. Finally the investigation of the body weight QTL syntenic regions and their reported QTLs are presented