Improving Facial Analysis and Performance Driven Animation through Disentangling Identity and Expression

We present techniques for improving performance driven facial animation, emotion recognition, and facial key-point or landmark prediction using learned identity invariant representations. Established approaches to these problems can work well if sufficient examples and labels for a particular identity are available and factors of variation are highly controlled. However, labeled examples of facial expressions, emotions and key-points for new individuals are difficult and costly to obtain. In this paper we improve the ability of techniques to generalize to new and unseen individuals by explicitly modeling previously seen variations related to identity and expression. We use a weakly-supervised approach in which identity labels are used to learn the different factors of variation linked to identity separately from factors related to expression. We show how probabilistic modeling of these sources of variation allows one to learn identity-invariant representations for expressions which can then be used to identity-normalize various procedures for facial expression analysis and animation control. We also show how to extend the widely used techniques of active appearance models and constrained local models through replacing the underlying point distribution models which are typically constructed using principal component analysis with identity-expression factorized representations. We present a wide variety of experiments in which we consistently improve performance on emotion recognition, markerless performance-driven facial animation and facial key-point tracking.Comment: to appear in Image and Vision Computing Journal (IMAVIS

Semantic Object Parsing with Graph LSTM

By taking the semantic object parsing task as an exemplar application scenario, we propose the Graph Long Short-Term Memory (Graph LSTM) network, which is the generalization of LSTM from sequential data or multi-dimensional data to general graph-structured data. Particularly, instead of evenly and fixedly dividing an image to pixels or patches in existing multi-dimensional LSTM structures (e.g., Row, Grid and Diagonal LSTMs), we take each arbitrary-shaped superpixel as a semantically consistent node, and adaptively construct an undirected graph for each image, where the spatial relations of the superpixels are naturally used as edges. Constructed on such an adaptive graph topology, the Graph LSTM is more naturally aligned with the visual patterns in the image (e.g., object boundaries or appearance similarities) and provides a more economical information propagation route. Furthermore, for each optimization step over Graph LSTM, we propose to use a confidence-driven scheme to update the hidden and memory states of nodes progressively till all nodes are updated. In addition, for each node, the forgets gates are adaptively learned to capture different degrees of semantic correlation with neighboring nodes. Comprehensive evaluations on four diverse semantic object parsing datasets well demonstrate the significant superiority of our Graph LSTM over other state-of-the-art solutions.Comment: 18 page

Oscillations in the Primordial Bispectrum: Mode Expansion

We consider the presence of oscillations in the primordial bispectrum, inspired by three different cosmological models; features in the primordial potential, resonant type non-Gaussianities and deviation from the standard Bunch Davies vacuum. In order to put constraints on their bispectra, a logical first step is to put these into factorized form which can be achieved via the recently proposed method of polynomial basis expansion on the tetrahedral domain. We investigate the viability of such an expansion for the oscillatory bispectra and find that one needs an increasing number of orthonormal mode functions to achieve significant correlation between the expansion and the original spectrum as a function of their frequency. To reduce the number of modes required, we propose a basis consisting of Fourier functions orthonormalized on the tetrahedral domain. We show that the use of Fourier mode functions instead of polynomial mode functions can lead to the necessary factorizability with the use of only 1/5 of the total number of modes required to reconstruct the bispectra with polynomial mode functions. Moreover, from an observational perspective, the expansion has unique signatures depending on the orientation of the oscillation due to a resonance effect between the mode functions and the original spectrum. This effect opens the possibility to extract informa- tion about both the frequency of the bispectrum as well as its shape while considering only a limited number of modes. The resonance effect is independent of the phase of the reconstructed bispectrum suggesting Fourier mode extraction could be an efficient way to detect oscillatory bispectra in the data.Comment: 17 pages, 12 figures. Matches published versio

Structural aspects of the fermion-boson mapping in two-dimensional gauge and anomalous gauge theories with massive fermions

Using a synthesis of the functional integral and operator approaches we discuss the fermion-boson mapping and the role played by the Bose field algebra in the Hilbert space of two-dimensional gauge and anomalous gauge field theories with massive fermions. In the $QED_2$ with quartic self-interaction among massive fermions, the use of an auxiliary vector field introduces a redundant Bose field algebra that should not be considered as an element of the intrinsic algebraic structure defining the model. In the anomalous chiral $QED_2$ with massive fermions the effect of the chiral anomaly leads to the appearance in the mass operator of a spurious Bose field combination. This phase factor carries no fermion selection rule and the expected absence of $\theta$-vacuum in the anomalous model is displayed from the operator solution. Even in the anomalous model with massive Fermi fields, the introduction of the Wess-Zumino field replicates the theory, changing neither its algebraic content nor its physical content.Comment: 26 pages, Revte

Entanglement and symmetry effects in the transition to the Schroedinger cat regime

We study two-spin entanglement and order parameter fluctuations as a function of the system size in the XY model in a transverse field and in the isotropic XXX model. Both models are characterized by the occurrence of ground state degeneracy also when systems of finite size are considered. This is always true for the XXX model, but only at the factorizing field for the XY model. We study the size dependence of symmetric states, which, in the presence of degeneracy, can be expanded as a linear combination of broken symmetry states. We show that, while the XY model looses its quantum superposition content exponentially with the size $N$, a decrease of the order of 1/N is observed when the XXX model is considered. The emergence of two qualitatively different regimes is directly related to the difference in the symmetry of the models

An exactly solvable dissipative transport model

We introduce a class of one-dimensional lattice models in which a quantity, that may be thought of as an energy, is either transported from one site to a neighbouring one, or locally dissipated. Transport is controlled by a continuous bias parameter q, which allows us to study symmetric as well as asymmetric cases. We derive sufficient conditions for the factorization of the N-body stationary distribution and give an explicit solution for the latter, before briefly discussing physically relevant situations.Comment: 7 pages, 1 figure, submitted to J. Phys.