A new framework of human interaction recognition based on multiple stage probability fusion

Visual-based human interactive behavior recognition is a challenging research topic in computer vision. There exist some important problems in the current interaction recognition algorithms, such as very complex feature representation and inaccurate feature extraction induced by wrong human body segmentation. In order to solve these problems, a novel human interaction recognition method based on multiple stage probability fusion is proposed in this paper. According to the human body’s contact in interaction as a cut-off point, the process of the interaction can be divided into three stages: start stage, execution stage and end stage. Two persons’ motions are respectively extracted and recognizes in the start stage and the finish stage when there is no contact between those persons. The two persons’ motion is extracted as a whole and recognized in the execution stage. In the recognition process, the final recognition results are obtained by the weighted fusing these probabilities in different stages. The proposed method not only simplifies the extraction and representation of features, but also avoids the wrong feature extraction caused by occlusion. Experiment results on the UT-interaction dataset demonstrated that the proposed method results in a better performance than other recent interaction recognition methods

Does M31 result from an ancient major merger?

The numerous streams in the M31 halo are currently assumed to be due to multiple minor mergers. Here we use the GADGET2 simulation code to test whether M31 could have experienced a major merger in its past history. It results that a 3+/-0.5:1 gaseous rich merger with r(per)=25+/-5 kpc and a polar orbit can explain many properties of M31 and of its halo. The interaction and the fusion may have begun 8.75+/-0.35 Gyr and 5.5 +/-0.5 Gyr ago, respectively. With an almost quiescent star formation history before the fusion we retrieve fractions of bulge, thin and thick disks as well as relative fractions of intermediate age and old stars in both the thick disk and the Giant Stream. The Giant Stream is caused by returning stars from a tidal tail previously stripped from the satellite prior to the fusion. These returning stars are trapped into elliptical orbits or loops for almost a Hubble time period. Large loops are also predicted and they scale rather well with the recently discovered features in the M31 outskirts. We demonstrate that a single merger could explain first-order (intensity and size), morphological and kinematical properties of the disk, thick disk, bulge and streams in the halo of M31, as well as the distribution of stellar ages, and perhaps metallicities. It challenges scenarios assuming one minor merger per feature in the disk (10 kpc ring) or at the outskirts (numerous streams & thick disk). Further constraints will help to properly evaluate the impact of such a major event to the Local Group.Comment: accepted in Astrophysical Journal, 29 September, 2010 ; proof-edited version; 1st column of Table 3 correcte

Mining multimodal sequential patterns : a case study on affect detection

Temporal data from multimodal interaction such as speech and bio-signals cannot be easily analysed without a preprocessing phase through which some key characteristics of the signals are extracted. Typically, standard statistical signal features such as average values are calculated prior to the analysis and, subsequently, are presented either to a multimodal fusion mechanism or a computational model of the interaction. This paper proposes a feature extraction methodology which is based on frequent sequence mining within and across multiple modalities of user input. The proposed method is applied for the fusion of physiological signals and gameplay information in a game survey dataset. The obtained sequences are analysed and used as predictors of user affect resulting in computational models of equal or higher accuracy compared to the models built on standard statistical features.peer-reviewe

Deep Affordance-grounded Sensorimotor Object Recognition

It is well-established by cognitive neuroscience that human perception of objects constitutes a complex process, where object appearance information is combined with evidence about the so-called object "affordances", namely the types of actions that humans typically perform when interacting with them. This fact has recently motivated the "sensorimotor" approach to the challenging task of automatic object recognition, where both information sources are fused to improve robustness. In this work, the aforementioned paradigm is adopted, surpassing current limitations of sensorimotor object recognition research. Specifically, the deep learning paradigm is introduced to the problem for the first time, developing a number of novel neuro-biologically and neuro-physiologically inspired architectures that utilize state-of-the-art neural networks for fusing the available information sources in multiple ways. The proposed methods are evaluated using a large RGB-D corpus, which is specifically collected for the task of sensorimotor object recognition and is made publicly available. Experimental results demonstrate the utility of affordance information to object recognition, achieving an up to 29% relative error reduction by its inclusion.Comment: 9 pages, 7 figures, dataset link included, accepted to CVPR 201

A Deep Spatio-Temporal Fuzzy Neural Network for Passenger Demand Prediction

In spite of its importance, passenger demand prediction is a highly challenging problem, because the demand is simultaneously influenced by the complex interactions among many spatial and temporal factors and other external factors such as weather. To address this problem, we propose a Spatio-TEmporal Fuzzy neural Network (STEF-Net) to accurately predict passenger demands incorporating the complex interactions of all known important factors. We design an end-to-end learning framework with different neural networks modeling different factors. Specifically, we propose to capture spatio-temporal feature interactions via a convolutional long short-term memory network and model external factors via a fuzzy neural network that handles data uncertainty significantly better than deterministic methods. To keep the temporal relations when fusing two networks and emphasize discriminative spatio-temporal feature interactions, we employ a novel feature fusion method with a convolution operation and an attention layer. As far as we know, our work is the first to fuse a deep recurrent neural network and a fuzzy neural network to model complex spatial-temporal feature interactions with additional uncertain input features for predictive learning. Experiments on a large-scale real-world dataset show that our model achieves more than 10% improvement over the state-of-the-art approaches.Comment: https://epubs.siam.org/doi/abs/10.1137/1.9781611975673.1