Automatic recognition of facial expressions

Facial expression is a visible manifestation of the affective state, cognitive activity, intention, personality and psychopathology of a person; it not only expresses our expressions, but also provides important communicative cues during social interaction. Expression recognition can be embedded into a face recognition system to improve its robustness. In a real-time face recognition system where a series of images of an individual are captured, facial expression recognition (FER) module picks the one which is most similar to a neutral expression for recognition, because normally a face recognition system is trained using neutral expression images. In the case where only one image is available, the estimated expression can be used either to decide which classifier to choose or to add some kind of compensation. In a human-computer interaction (HCI), expression is an input of great potential in terms of communicative cues. This is especially true in voice-activated control systems. This implies an FER module can markedly improve the performance of such systems. Customer's facial expressions can also be collected by service providers as implicit user feedback to improve their service. Compared with a conventional questionnaire-based method, this should be more reliable and furthermore, has virtually no cost. The main challenge for FER system is to attain the highest possible classification rate for the recognition of six expressions (Anger, Disgust, Fear, Happy, Sad and Surprise). The other challenges are the illumination variation, rotation and noise. In this thesis, several innovative methods based on image processing and pattern recognition theory have been devised and implemented. The main contributions of algorithms and advanced modelling techniques are summarized as follows. 1) A new feature extraction approach called HLAC-like (higher-order local autocorrelation-like) features has been presented to detect and to extract facial features from face images. 2) An innovative design is introduced with the ability to detect cases using face feature extraction method based on orthogonal moments for images with noise and/or rotation. Using this technique, the expression from face images with high levels of noise and even rotation has been recognized properly. 3) A facial expression recognition system is designed based on the combination region. In this system, a method called hybrid face regions (HFR) according to the combined part of an image is presented. Using this method, the features are extracted from the components of the face (eyes, nose and mouth) and then the expression is identified based on these features. 4) A novel classification methodology has been proposed based on structural similarity algorithm in facial expression recognition scenarios. 5) A new methodology for expression recognition is presented using colour facial images based on multi-linear image analysis. In this scenario, the colour images are unfolded to two dimensional (2-D) matrix based on multi-linear algebra and then classified based on multi-linear discriminant analysis (LDA) classifier. Furthermore, the colour effect on facial images of various resolutions is studied for FER system. The addressed issues are challenging problems and are substantial for developing a facial expression recognition system

Railway reinforced concrete infrastructure life management and sustainability index

Infrastructure healthy enhancement for saving resources in operation procedures is one of the most important objectives for owners on their decision support system based on cost management. In this manner, finding the intervention action priority, as well as the inspection method and maintenance system for each component, with regard to a limited resources amount is investigated in this paper. Defects on infrastructure components create data and these data are undoubtedly useful to increase the knowledge in decision making in practice. In that sense, risk analysis and value of information can be applied using decision trees together with Bayesian networks. For data filtering and noise reduction, a principal component analysis may also be applied to manage a database and prepare useful input variables for the decision tree system. This paper presented an approach for the maintenance managers to prepare their infrastructure available with a sustainable index with minimum cost. This index would be a tool for decision-makers with regard to the cost management and sustainability aspects

Operation reliability index for maintenance decision making in bridges

Maintenance management developed by several approach to optimize cost recently. Meanwhile decision making during operation is difficult task for mangers to keep them safe as well as stakeholder demands satisfaction and costs with regard to resources limitation. This paper presents an approach for decision making process to select alternatives based on their costs. For this manner, the uncertainty of defect probability combine with other availability and performance features to find priority of maintenance equipment and their reliability. This multi-dimensional decision making do not deal with the essential imprecision of subjective judgment based on quantitative evaluation. To demonstrate the use and capability of the model, a case study is presented. In this case, results shows the quality value combined by delay as an effectiveness parameters (91.08) and then decision tree will complete it by risk and reliability factors

Occupational challenges in the caregivers of people with multiple sclerosis: A qualitative study

Background: Multiple Sclerosis (MS) is a neurodegenerative disorder. The progressive and unpredictable nature of MS indicates the patients� need for long-term care as well as the increased burden of their primary caregivers� care and occupational challenges that emerge in their daily life activities. Objectives: The current study aimed to explore the occupational challenges caused by engaging in the care process for the caregivers of people with MS Methods: This qualitative study was conducted on 21 caregivers of MS patients using a content analysis approach. Data were collected through semi-structured face-to-face interviews. Results: Three main themes emerged: Time limitations in occupation implementation, care needs in occupation implementation, and emotional reactions affecting occupations. Conclusions: According to the results, the caregivers of people with MS were faced with a variety of occupational challenges. The time limitations for performing routine occupations and desired activities, unmet patient care-facilitating needs, and the occurrence of psychosocial reactions and behaviors were contributed to these challenges and their exacerbation. Identifying these challenges is both useful for designing interventional programs and to help caregivers to successfully perform their desired occupations in spite of challenges in the care process. © 2020, Semnan University of Medical Sciences. All rights reserved

Structural similarity classifier for facial expression recognition

A novel Gabor filter structural similarity algorithm (GFSSIM) is proposed for facial expression recognition (FER) on noisy images. Low-resolution facial images with low SNRs are specifically dealt with FER system. The features are extracted using 40 Gabor filters, and a feature subset is selected for classification. The test image is classified based on proposed GFSSIM algorithm. The experimental results show that the recognition rate for heavily deteriorated images outperforms the conventional classifier method. In addition, the proposed method is more efficient from the computational complexity point of view

Local feature extraction methods for facial expression recognition

In this paper we investigate the performance of different feature extraction methods for facial expression recognition based on the higher-order local autocorrelation (HLAC) coefficients and local binary pattern (LBP) operator. Autocorrelation coefficients are computationally inexpensive, inherently shift-invariant and quite robust against changes in facial expression. The focus is on the difficult problem of recognizing an expression in different resolutions. Results indicate that LBP coefficients have surprisingly high information content

Facial expression recognition using log-Gabor filters and local binary pattern operators

This study investigates two different methods of feature extraction for person-independent facial expression recognition from images. The logarithmic Gabor filters and the local binary pattern operator (LBP) were used for feature extraction. Then, the optimum features were selected based on minimum redundancy and maximum relevance algorithm (MRMR). Six different facial expressions were considered. The selected features were classified using the naive bayesian (NB) classifier. The percentage of correct classifications varied across different expressions from 62.8% to 90.5% for the log-Gabor filter approach,and from 71.8% to 94.0% for the LBP approach. Experiments carried out on Cohn-Kanade database showed comparable performance between Log-Gabor filters and LBP operator, with a classification accuracy of around 82.3% and 81.7% respectively. This was achieved on low-resolution images, without the need to precisely locate facial points on each face image

Averaged Gabor filter features for facial expression recognition

An efficient automatic facial expression recognition method is proposed. The method uses a set of characteristic features obtained by averaging the outputs from the Gabor filter bank with 5 frequencies and 8 different orientations, and then further reducing the dimensionality by the means of principal component analysis. The performance of the proposed system was compared with the full Gabor filter bank method. The classification tasks were performed using the K-Nearest neighbor (K-NN) classifier. The training and testing images were selected from the publicly available JAFFE database. The classification results show that the average Gabor filter (AGF) provides very high computational efficiency at the cost of a relatively small decrease in performance when compared to the full Gabor filter features

Higher order orthogonal moments for invariant facial expression recognition

Automatic facial expression recognition (FER) is a sub-area of face analysis research that is based heavily on methods of computer vision, machine learning, and image processing. This study proposes a rotation and noise invariant FER system using an orthogonal invariant moment, namely, Zernike moments (ZM) as a feature extractor and Naive Bayesian (NB) classifier. The system is fully automatic and can recognize seven different expressions. Illumination condition, pose, rotation, noise and others changing in the image are challenging task in pattern recognition system. Simulation results on different databases indicated that higher order ZM features are robust in images that are affected by noise and rotation, whereas the computational rate for feature extraction is lower than other methods