Joint Alignment and Modeling of Correlated Behavior Streams

The Variable Time-Shift Hidden Markov Model (VTS- HMM) is proposed for learning and modeling pairs of cor- related streams. Unlike previous coupled models for time series, the VTS-HMM accounts for varying time shifts be- tween correlated events in pairs of streams having different properties. The VTS-HMM is learned on a set of pairs of unaligned streams and, thus, learning entails simultaneous estimation of the varying time shifts and of the parameters of the model. The formulation is demonstrated in the analysis of videos of dyadic social interactions between children and adults in the Multimodal Dyadic Behavior Dataset (MMDB). In dyadic social interactions, an agent starts an interaction with one or more \u201cinitiating behaviors\u201d that elicit one or more \u201cresponding behaviors\u201d from the partner within a temporal window. The proposed VTS-HMM explicitly accounts for varying time shifts between initiating and responding behaviors in these behavior streams. The experiments confirm that modeling of these varying time shifts in the VTS-HMM can yield improved estimation of the level of engagement of the child and adult and more accurate dis- crimination among complex activities

Joint perceptual decision-making: a case study in explanatory pluralism.

Traditionally different approaches to the study of cognition have been viewed as competing explanatory frameworks. An alternative view, explanatory pluralism, regards different approaches to the study of cognition as complementary ways of studying the same phenomenon, at specific temporal and spatial scales, using appropriate methodological tools. Explanatory pluralism has been often described abstractly, but has rarely been applied to concrete cases. We present a case study of explanatory pluralism. We discuss three separate ways of studying the same phenomenon: a perceptual decision-making task (Bahrami et al., 2010), where pairs of subjects share information to jointly individuate an oddball stimulus among a set of distractors. Each approach analyzed the same corpus but targeted different units of analysis at different levels of description: decision-making at the behavioral level, confidence sharing at the linguistic level, and acoustic energy at the physical level. We discuss the utility of explanatory pluralism for describing this complex, multiscale phenomenon, show ways in which this case study sheds new light on the concept of pluralism, and highlight good practices to critically assess and complement approaches

MX-LSTM: mixing tracklets and vislets to jointly forecast trajectories and head poses

Recent approaches on trajectory forecasting use tracklets to predict the future positions of pedestrians exploiting Long Short Term Memory (LSTM) architectures. This paper shows that adding vislets, that is, short sequences of head pose estimations, allows to increase significantly the trajectory forecasting performance. We then propose to use vislets in a novel framework called MX-LSTM, capturing the interplay between tracklets and vislets thanks to a joint unconstrained optimization of full covariance matrices during the LSTM backpropagation. At the same time, MX-LSTM predicts the future head poses, increasing the standard capabilities of the long-term trajectory forecasting approaches. With standard head pose estimators and an attentional-based social pooling, MX-LSTM scores the new trajectory forecasting state-of-the-art in all the considered datasets (Zara01, Zara02, UCY, and TownCentre) with a dramatic margin when the pedestrians slow down, a case where most of the forecasting approaches struggle to provide an accurate solution.Comment: 10 pages, 3 figures to appear in CVPR 201

Four phases of angular-momentum buildup in high-z galaxies: from cosmic-web streams through an extended ring to disc and bulge

We study the angular-momentum (AM) buildup in high-$z$ massive galaxies using high-resolution cosmological simulations. The AM originates in co-planar streams of cold gas and merging galaxies tracing cosmic-web filaments, and it undergoes four phases of evolution. (I) Outside the halo virial radius ($R_{\rm v}\!\sim\!100\,{\rm kpc}$), the elongated streams gain AM by tidal torques with a specific AM (sAM) $\sim\!1.7$ times the dark-matter (DM) spin due to the gas' higher quadrupole moment. This AM is expressed as stream impact parameters, from $\sim\!0.3R_{\rm v}$ to counter rotation. (II) In the outer halo, while the incoming DM mixes with the existing halo of lower sAM to a spin $\lambda_{\rm dm}\!\sim\!0.04$, the cold streams transport the AM to the inner halo such that their spin in the halo is $\sim\!3\lambda_{\rm dm}$. (III) Near pericenter, the streams dissipate into an irregular rotating ring extending to $\sim\!0.3R_{\rm v}$ and tilted relative to the inner disc. Torques exerted partly by the disc make the ring gas lose AM, spiral in, and settle into the disc within one orbit. The ring is observable with 30\% probability as a damped Lyman-$\alpha$ absorber. (IV) Within the disc, $<\!0.1R_{\rm v}$, torques associated with violent disc instability drive AM out and baryons into a central bulge, while outflows remove low-spin gas, introducing certain sensitivity to feedback strength. Despite the different AM histories of gas and DM, the disc spin is comparable to the DM-halo spin. Counter rotation can strongly affect disc evolution.Comment: Resubmitted to MNRAS after responding to referee's comments. (27 pages, 20 figures

Ensemble of Hankel Matrices for Face Emotion Recognition

In this paper, a face emotion is considered as the result of the composition of multiple concurrent signals, each corresponding to the movements of a specific facial muscle. These concurrent signals are represented by means of a set of multi-scale appearance features that might be correlated with one or more concurrent signals. The extraction of these appearance features from a sequence of face images yields to a set of time series. This paper proposes to use the dynamics regulating each appearance feature time series to recognize among different face emotions. To this purpose, an ensemble of Hankel matrices corresponding to the extracted time series is used for emotion classification within a framework that combines nearest neighbor and a majority vote schema. Experimental results on a public available dataset shows that the adopted representation is promising and yields state-of-the-art accuracy in emotion classification.Comment: Paper to appear in Proc. of ICIAP 2015. arXiv admin note: text overlap with arXiv:1506.0500