IMPROVING ROBUSTNESS OF FACE EXPRESSION RECOGNITION USING MULTI-CHANNEL LOCAL BINARY PATTERN AND NEURAL NETWORK

ABSTRACTFacial Expression Recognition (FER) is a subset of Artificial Intelligence (AI) that relates to human non-verbal communication. The development of Convolutional Neural Network (CNN) based FER is subject to noise, mainly because of the usage of RGB Original Image as training data. Many research explored texture feature methods which noise resistant, such as Local Binary Pattern (LBP) and Gray Level Co-occurrence Matrix (GLCM), which mainly worked on grayscale images. Multi-Channel Local Binary Pattern (MCLBP) is derived from LBP which analyzes texture on color images.This research aims to develop FER using MCLBP as a method of hand-crafted texture feature and NN as a classification method. The combination of MCLBP and Neural Network (NN) is expected more robust to noise. First, preprocessing is applied to the facial image for contrasting with Adaptive Gamma Correction Weighted Distribution (AGCWD). Next, the facial image is converted to MCLBP images. Then MCLBP images are converted to vectors as a NN architecture training data with 5 Fully Connected layers. Batch Normalization and Rectified Linear Unit (ReLu) activation are used in every Fully Connected layer. At the last Fully Connected Layer, ReLu activation was replaced with SoftMax activation. This NN uses Stochastic Gradient Descend (SGD) optimizer with a learning rate of 0.005.Performance testing was held by comparing the epoch required to reach F1-score 1 and F1-Score from many scenarios in FER with LBP + NN with 140 × 190 image size, LBP + NN with 70 × 85 image size, and MCLBP + NN with 70 × 85 image size approaches. From all scenarios we have tried, the best method is MCLBP with F1-Score =1 in 22 epochs. The method of hand-crafted texture feature with NN can increase the desirable FER performances.                                                                                       Keywords: Local Binary Pattern, Multi-Channel LBP, Neural Network, Face Expression Recognition, Gamma Correctio

A Novel Adaptive LBP-Based Descriptor for Color Image Retrieval

In this paper, we present two approaches to extract discriminative features for color image retrieval. The proposed local texture descriptors, based on Radial Mean Local Binary Pattern (RMLBP), are called Color RMCLBP (CRMCLBP) and Prototype Data Model (PDM). RMLBP is a robust to noise descriptor which has been proposed to extract texture features of gray scale images for texture classification. For the first descriptor, the Radial Mean Completed Local Binary Pattern is applied to channels of the color space, independently. Then, the final descriptor is achieved by concatenating the histogram of the CRMCLBP_S/M/C component of each channel. Moreover, to enhance the performance of the proposed method, the Particle Swarm Optimization (PSO) algorithm is used for feature weighting. The second proposed descriptor, PDM, uses the three outputs of CRMCLBP (CRMCLBP_S, CRMCLBP_M, CRMCLBP_C) as discriminative features for each pixel of a color image. Then, a set of representative feature vectors are selected from each image by applying k-means clustering algorithm. This set of selected prototypes are compared by means of a new similarity measure to find the most relevant images. Finally, the weighted versions of PDM is constructed using PSO algorithm. Our proposed methods are tested on Wang, Corel-5k, Corel-10k and Holidays datasets. The results show that our proposed methods makes an admissible tradeoff between speed and retrieval accuracy. The first descriptor enhances the state-of-the-art color texture descriptors in both aspects. The second one is a very fast retrieval algorithm which extracts discriminative features

Feature-based image patch classification for moving shadow detection

Moving object detection is a first step towards many computer vision applications, such as human interaction and tracking, video surveillance, and traffic monitoring systems. Accurate estimation of the target object’s size and shape is often required before higher-level tasks (e.g., object tracking or recog nition) can be performed. However, these properties can be derived only when the foreground object is detected precisely. Background subtraction is a common technique to extract foreground objects from image sequences. The purpose of background subtraction is to detect changes in pixel values within a given frame. The main problem with background subtraction and other related object detection techniques is that cast shadows tend to be misclassified as either parts of the foreground objects (if objects and their cast shadows are bonded together) or independent foreground objects (if objects and shadows are separated). The reason for this phenomenon is the presence of similar characteristics between the target object and its cast shadow, i.e., shadows have similar motion, attitude, and intensity changes as the moving objects that cast them. Detecting shadows of moving objects is challenging because of problem atic situations related to shadows, for example, chromatic shadows, shadow color blending, foreground-background camouflage, nontextured surfaces and dark surfaces. Various methods for shadow detection have been proposed in the liter ature to address these problems. Many of these methods use general-purpose image feature descriptors to detect shadows. These feature descriptors may be effective in distinguishing shadow points from the foreground object in a specific problematic situation; however, such methods often fail to distinguish shadow points from the foreground object in other situations. In addition, many of these moving shadow detection methods require prior knowledge of the scene condi tions and/or impose strong assumptions, which make them excessively restrictive in practice. The aim of this research is to develop an efficient method capable of addressing possible environmental problems associated with shadow detection while simultaneously improving the overall accuracy and detection stability. In this research study, possible problematic situations for dynamic shad ows are addressed and discussed in detail. On the basis of the analysis, a ro bust method, including change detection and shadow detection, is proposed to address these environmental problems. A new set of two local feature descrip tors, namely, binary patterns of local color constancy (BPLCC) and light-based gradient orientation (LGO), is introduced to address the identified problematic situations by incorporating intensity, color, texture, and gradient information. The feature vectors are concatenated in a column-by-column manner to con struct one dictionary for the objects and another dictionary for the shadows. A new sparse representation framework is then applied to find the nearest neighbor of the test image segment by computing a weighted linear combination of the reference dictionary. Image segment classification is then performed based on the similarity between the test image and the sparse representations of the two classes. The performance of the proposed framework on common shadow detec tion datasets is evaluated, and the method shows improved performance com pared with state-of-the-art methods in terms of the shadow detection rate, dis crimination rate, accuracy, and stability. By achieving these significant improve ments, the proposed method demonstrates its ability to handle various problems associated with image processing and accomplishes the aim of this thesis

Robust approaches for face recognition

This thesis gave answers to a number of important questions regarding face classification. Via this research, new methods were introduced to represent four facial attributes (three of them related to the demographic information of the human face: gender, age and race) and the fourth one related to facial expression. It stated that, discriminative facial features regarding to demographic information (gender, age and race) and expression information can be obtained by applying texture analysis techniques to the polar raster sampled images. In addition, it is found that, multi-label classification (MLC) is more suitable in the real world as a human face can be associated with multiple labels