Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories

In this paper, we propose a new approach for facial expression recognition using deep covariance descriptors. The solution is based on the idea of encoding local and global Deep Convolutional Neural Network (DCNN) features extracted from still images, in compact local and global covariance descriptors. The space geometry of the covariance matrices is that of Symmetric Positive Definite (SPD) matrices. By conducting the classification of static facial expressions using Support Vector Machine (SVM) with a valid Gaussian kernel on the SPD manifold, we show that deep covariance descriptors are more effective than the standard classification with fully connected layers and softmax. Besides, we propose a completely new and original solution to model the temporal dynamic of facial expressions as deep trajectories on the SPD manifold. As an extension of the classification pipeline of covariance descriptors, we apply SVM with valid positive definite kernels derived from global alignment for deep covariance trajectories classification. By performing extensive experiments on the Oulu-CASIA, CK+, and SFEW datasets, we show that both the proposed static and dynamic approaches achieve state-of-the-art performance for facial expression recognition outperforming many recent approaches.Comment: A preliminary version of this work appeared in "Otberdout N, Kacem A, Daoudi M, Ballihi L, Berretti S. Deep Covariance Descriptors for Facial Expression Recognition, in British Machine Vision Conference 2018, BMVC 2018, Northumbria University, Newcastle, UK, September 3-6, 2018. ; 2018 :159." arXiv admin note: substantial text overlap with arXiv:1805.0386

An Emergent Space for Distributed Data with Hidden Internal Order through Manifold Learning

Manifold-learning techniques are routinely used in mining complex spatiotemporal data to extract useful, parsimonious data representations/parametrizations; these are, in turn, useful in nonlinear model identification tasks. We focus here on the case of time series data that can ultimately be modelled as a spatially distributed system (e.g. a partial differential equation, PDE), but where we do not know the space in which this PDE should be formulated. Hence, even the spatial coordinates for the distributed system themselves need to be identified - to emerge from - the data mining process. We will first validate this emergent space reconstruction for time series sampled without space labels in known PDEs; this brings up the issue of observability of physical space from temporal observation data, and the transition from spatially resolved to lumped (order-parameter-based) representations by tuning the scale of the data mining kernels. We will then present actual emergent space discovery illustrations. Our illustrative examples include chimera states (states of coexisting coherent and incoherent dynamics), and chaotic as well as quasiperiodic spatiotemporal dynamics, arising in partial differential equations and/or in heterogeneous networks. We also discuss how data-driven spatial coordinates can be extracted in ways invariant to the nature of the measuring instrument. Such gauge-invariant data mining can go beyond the fusion of heterogeneous observations of the same system, to the possible matching of apparently different systems

A Novel Representation for Two-dimensional Image Structures

This paper presents a novel approach towards two-dimensional (2D) image structures modeling. To obtain more degrees of freedom, a 2D image signal is embedded into a certain geometric algebra. Coupling methods of differential geometry, tensor algebra, monogenic signal and quadrature filter, we can design a general model for 2D structures as the monogenic extension of a curvature tensor. Based on it, a local representation for the intrinsically two-dimensional (i2D) structure is derived as the monogenic curvature signal. From it, independent features of local amplitude, phase and orientation are simultaneously extracted. Besides, a monogenic curvature scale-space can be built by applying a Poisson kernel to the monogenic curvature signal. Compared with the other related work, the remarkable advantage of our approach lies in the rotationally invariant phase evaluation of 2D structures in a multi-scale framework, which delivers access to phase-based processing in many computer vision tasks

Volumetric Untrimming: Precise decomposition of trimmed trivariates into tensor products

3D objects, modeled using Computer Aided Geometric Design tools, are traditionally represented using a boundary representation (B-rep), and typically use spline functions to parameterize these boundary surfaces. However, recent development in physical analysis, in isogeometric analysis (IGA) in specific, necessitates a volumetric parametrization of the interior of the object. IGA is performed directly by integrating over the spline spaces of the volumetric spline representation of the object. Typically, tensor-product B-spline trivariates are used to parameterize the volumetric domain. A general 3D object, that can be modeled in contemporary B-rep CAD tools, is typically represented using trimmed B-spline surfaces. In order to capture the generality of the contemporary B-rep modeling space, while supporting IGA needs, Massarwi and Elber (2016) proposed the use of trimmed trivariates volumetric elements. However, the use of trimmed geometry makes the integration process more difficult since integration over trimmed B-spline basis functions is a highly challenging task. In this work, we propose an algorithm that precisely decomposes a trimmed B-spline trivariate into a set of (singular only on the boundary) tensor-product B-spline trivariates, that can be utilized to simplify the integration process in IGA. The trimmed B-spline trivariate is first subdivided into a set of trimmed B\'ezier trivariates, at all its internal knots. Then, each trimmed B\'ezier trivariate, is decomposed into a set of mutually exclusive tensor-product B-spline trivariates, that precisely cover the entire trimmed domain. This process, denoted untrimming, can be performed in either the Euclidean space or the parametric space of the trivariate. We present examples on complex trimmed trivariates' based geometry, and we demonstrate the effectiveness of the method by applying IGA over the (untrimmed) results.Comment: 18 pages, 32 figures. Contribution accepted in International Conference on Geometric Modeling and Processing (GMP 2019

Geometric deep learning: going beyond Euclidean data

Many scientific fields study data with an underlying structure that is a non-Euclidean space. Some examples include social networks in computational social sciences, sensor networks in communications, functional networks in brain imaging, regulatory networks in genetics, and meshed surfaces in computer graphics. In many applications, such geometric data are large and complex (in the case of social networks, on the scale of billions), and are natural targets for machine learning techniques. In particular, we would like to use deep neural networks, which have recently proven to be powerful tools for a broad range of problems from computer vision, natural language processing, and audio analysis. However, these tools have been most successful on data with an underlying Euclidean or grid-like structure, and in cases where the invariances of these structures are built into networks used to model them. Geometric deep learning is an umbrella term for emerging techniques attempting to generalize (structured) deep neural models to non-Euclidean domains such as graphs and manifolds. The purpose of this paper is to overview different examples of geometric deep learning problems and present available solutions, key difficulties, applications, and future research directions in this nascent field