Analysis of Range Images Used in 3D Facial Expression Recognition Systems

With the creation of BU-3DFE database the research on 3D facial expression recognition has been fostered; however, it is limited by the development of 3D algorithms. Range image is the strategy for solving the problems of 3D recognition based on 2D algorithms. Recently, there are some methods to capture range images, but they are always combined with the preprocess, registration, etc. stages, so it is hard to tell which of these generated range images is of higher quality. This paper introduces two kinds of range images and selects different kinds of features based on different levels of expressions to validate the performances of proposed range images; two other kinds of range images based on previously used nose tip detection methods are applied to compare the quality of generated range images; and finally some recently published works on 3D facial expression recognition are listed for comparison. With the experimental results, we can see that the performances of two proposed range images with different kinds of features are all higher than 88 % which is remarkable compared with the most recently published methods for 3D facial expression recognition; the analysis of the different kinds of facial expressions shows that the proposed range images do not lose primary discriminative information for recognition; the performances of range images using different kinds of nose tip detection methods are almost the same what means that the nose tip detection is not decisive to the quality of range images; moreover, the proposed range images can be captured without any manual intervention what is eagerly required in safety systems

The Dynamic Model Embed in Augmented Graph Cuts for Robust Hand Tracking and Segmentation in Videos

Segmenting human hand is important in computer vision applications, for example, sign language interpretation, human computer interaction, and gesture recognition. However, some serious bottlenecks still exist in hand localization systems such as fast hand motion capture, hand over face, and hand occlusions on which we focus in this paper. We present a novel method for hand tracking and segmentation based on augmented graph cuts and dynamic model. First, an effective dynamic model for state estimation is generated, which correctly predicts the location of hands probably having fast motion or shape deformations. Second, new energy terms are brought into the energy function to develop augmented graph cuts based on some cues, namely, spatial information, hand motion, and chamfer distance. The proposed method successfully achieves hand segmentation even though the hand passes over other skin-colored objects. Some challenging videos are provided in the case of hand over face, hand occlusions, dynamic background, and fast motion. Experimental results demonstrate that the proposed method is much more accurate than other graph cuts-based methods for hand tracking and segmentation

Subgraph and object context‐masked network for scene graph generation

Scene graph generation is to recognise objects and their semantic relationships in an image and can help computers understand visual scene. To improve relationship prediction, geometry information is essential and usually incorporated into relationship features. Existing methods use coordinates of objects to encode their spatial layout. However, in this way, they neglect the context of objects. In this study, to take full use of spatial knowledge efficiently, the authors propose a novel subgraph and object context‐masked network (SOCNet) consisting of spatial mask relation inference (SMRI) and hierarchical message passing (HMP) modules to address the scene graph generation task. In particular, to take advantage of spatial knowledge, SMRI masks partial context of object features depending on their spatial layout of objects and corresponding subgraph to facilitate their relationship recognition. To refine the features of objects and subgraphs, they also propose HMP that passes highly correlated messages from both microcosmic and macroscopic aspects through a triple‐path structure including subgraph–subgraph, object–object, and subgraph–object paths. Finally, statistical co‐occurrence probability is used to regularise relationship prediction. SOCNet integrates HMP and SMRI into a unified network, and comprehensive experiments on visual relationship detection and visual genome datasets indicate that SOCNet outperforms several state‐of‐the‐art methods on two common tasks

Orthogonal tucker decomposition using factor priors for 2D+3D facial expression recognition

Abstract In this article, an effective approach is proposed to recognise the 2D+3D facial expression automatically based on orthogonal Tucker decomposition using factor priors (OTDFPFER). As a powerful technique, Tucker decomposition on the basis of the low rank approximation is often used to extract the useful information from the constructed 4D tensor composed of 3D face scans and 2D images aiming to maintain correlations and their structural information. Finding a set of projected factor matrices is our ultimate goal. During the 4D tensor modelling process, high similarities among samples will emerge because of the information missed partially. Based on the tensor orthogonal Tucker decomposition, the involved core tensor with the structured sparsity, and a graph regularisation term via the graph Laplacian matrix together with the fourth factor matrix are employed for better characterisation of the generated similarities and for keeping the consistency of low dimensional space. To recover the missing information, a framework for tensor completion (TC) will be embedded naturally. Finally, an alternating direction method coupled with the majorisation‐minimisation scheme is designed to solve the resulting tensor completion problem. The numerical experiments are conducted on the Bosphorus and the BU‐3DFE databases with promising recognition accuracies

Facial Expression Recognition Based on Discriminant Neighborhood Preserving Nonnegative Tensor Factorization and ELM

A novel facial expression recognition algorithm based on discriminant neighborhood preserving nonnegative tensor factorization (DNPNTF) and extreme learning machine (ELM) is proposed. A discriminant constraint is adopted according to the manifold learning and graph embedding theory. The constraint is useful to exploit the spatial neighborhood structure and the prior defined discriminant properties. The obtained parts-based representations by our algorithm vary smoothly along the geodesics of the data manifold and have good discriminant property. To guarantee the convergence, the project gradient method is used for optimization. Then features extracted by DNPNTF are fed into ELM which is a training method for the single hidden layer feed-forward networks (SLFNs). Experimental results on JAFFE database and Cohn-Kanade database demonstrate that our proposed algorithm could extract effective features and have good performance in facial expression recognition