Multimodal Signal Processing and Learning Aspects of Human-Robot Interaction for an Assistive Bathing Robot

We explore new aspects of assistive living on smart human-robot interaction (HRI) that involve automatic recognition and online validation of speech and gestures in a natural interface, providing social features for HRI. We introduce a whole framework and resources of a real-life scenario for elderly subjects supported by an assistive bathing robot, addressing health and hygiene care issues. We contribute a new dataset and a suite of tools used for data acquisition and a state-of-the-art pipeline for multimodal learning within the framework of the I-Support bathing robot, with emphasis on audio and RGB-D visual streams. We consider privacy issues by evaluating the depth visual stream along with the RGB, using Kinect sensors. The audio-gestural recognition task on this new dataset yields up to 84.5%, while the online validation of the I-Support system on elderly users accomplishes up to 84% when the two modalities are fused together. The results are promising enough to support further research in the area of multimodal recognition for assistive social HRI, considering the difficulties of the specific task. Upon acceptance of the paper part of the data will be publicly available

Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognition

Over the past few years, deep learning methods have shown remarkable results in many face-related tasks including automatic facial expression recognition (FER) in-the-wild. Meanwhile, numerous models describing the human emotional states have been proposed by the psychology community. However, we have no clear evidence as to which representation is more appropriate and the majority of FER systems use either the categorical or the dimensional model of affect. Inspired by recent work in multi-label classification, this paper proposes a novel multi-task learning (MTL) framework that exploits the dependencies between these two models using a Graph Convolutional Network (GCN) to recognize facial expressions in-the-wild. Specifically, a shared feature representation is learned for both discrete and continuous recognition in a MTL setting. Moreover, the facial expression classifiers and the valence-arousal regressors are learned through a GCN that explicitly captures the dependencies between them. To evaluate the performance of our method under real-world conditions we perform extensive experiments on the AffectNet and Aff-Wild2 datasets. The results of our experiments show that our method is capable of improving the performance across different datasets and backbone architectures. Finally, we also surpass the previous state-of-the-art methods on the categorical model of AffectNet.Comment: 9 pages, 8 figures, 5 tables, revised submission to the 16th IEEE International Conference on Automatic Face and Gesture Recognitio

The Generalization of the Decomposition of Functions by Energy Operators

This work starts with the introduction of a family of differential energy operators. Energy operators ($Psi_R^+$, $Psi_R^-$) were defined together with a method to decompose the wave equation in a previous work. Here the energy operators are defined following the order of their derivatives ($Psi_k^+$, $Psi_k^-$, k = {0,1,2,..}). The main part of the work is to demonstrate that for any smooth real-valued function f in the Schwartz space ($S^-(R)$), the successive derivatives of the n-th power of f (n in Z and n not equal to 0) can be decomposed using only $Psi_k^+$ (Lemma) or with $Psi_k^+$, $Psi_k^-$ (k in Z) (Theorem) in a unique way (with more restrictive conditions). Some properties of the Kernel and the Image of the energy operators are given along with the development. Finally, the paper ends with the application to the energy function.Comment: The paper was accepted for publication at Acta Applicandae Mathematicae (15/05/2013) based on v3. v4 is very similar to v3 except that we modified slightly Definition 1 to make it more readable when showing the decomposition with the families of energy operator of the derivatives of the n-th power of

Structure tensor total variation

This is the final version of the article. Available from Society for Industrial and Applied Mathematics via the DOI in this record.We introduce a novel generic energy functional that we employ to solve inverse imaging problems within a variational framework. The proposed regularization family, termed as structure tensor total variation (STV), penalizes the eigenvalues of the structure tensor and is suitable for both grayscale and vector-valued images. It generalizes several existing variational penalties, including the total variation seminorm and vectorial extensions of it. Meanwhile, thanks to the structure tensor’s ability to capture first-order information around a local neighborhood, the STV functionals can provide more robust measures of image variation. Further, we prove that the STV regularizers are convex while they also satisfy several invariance properties w.r.t. image transformations. These properties qualify them as ideal candidates for imaging applications. In addition, for the discrete version of the STV functionals we derive an equivalent definition that is based on the patch-based Jacobian operator, a novel linear operator which extends the Jacobian matrix. This alternative definition allow us to derive a dual problem formulation. The duality of the problem paves the way for employing robust tools from convex optimization and enables us to design an efficient and parallelizable optimization algorithm. Finally, we present extensive experiments on various inverse imaging problems, where we compare our regularizers with other competing regularization approaches. Our results are shown to be systematically superior, both quantitatively and visually

On Close Relationship between Classical Time-Dependent Harmonic Oscillator and Non-Relativistic Quantum Mechanics in One Dimension

In this paper, I present a mapping between representation of some quantum phenomena in one dimension and behavior of a classical time-dependent harmonic oscillator. For the first time, it is demonstrated that quantum tunneling can be described in terms of classical physics without invoking violations of the energy conservation law at any time instance. A formula is presented that generates a wide class of potential barrier shapes with the desirable reflection (transmission) coefficient and transmission phase shift along with the corresponding exact solutions of the time-independent Schr\"odinger's equation. These results, with support from numerical simulations, strongly suggest that two uncoupled classical harmonic oscillators, which initially have a 90\degree relative phase shift and then are simultaneously disturbed by the same parametric perturbation of a finite duration, manifest behavior which can be mapped to that of a single quantum particle, with classical 'range relations' analogous to the uncertainty relations of quantum physics.Comment: 20 pages, 8 figures, 1 table, final versio