Action classification using a discriminative non-parametric hidden Markov model

We classify human actions occurring in videos, using the skeletal joint positions extracted from a depth image sequence as features. Each action class is represented by a non-parametric Hidden Markov Model (NP-HMM) and the model parameters are learnt in a discriminative way. Specifically, we use a Bayesian framework based on Hierarchical Dirichlet Process (HDP) to automatically infer the cardinality of hidden states and formulate a discriminative function based on distance between Gaussian distributions to improve classification performance. We use elliptical slice sampling to efficiently sample parameters from the complex posterior distribution induced by our discriminative likelihood function. We illustrate our classification results for action class models trained using this technique

Action Recognition in Videos: from Motion Capture Labs to the Web

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, "in the wild" videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4 table

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Endothelial cell phenotypic behaviors cluster into dynamic state transition programs modulated by angiogenic and angiostatic cytokines

Angiogenesis requires coordinated dynamic regulation of multiple phenotypic behaviors of endothelial cells in response to environmental cues. Multi-scale computational models of angiogenesis can be useful for analyzing effects of cell behaviors on the tissue level outcome, but these models require more intensive experimental studies dedicated to determining the required quantitative “rules” for cell-level phenotypic responses across a landscape of pro- and anti-angiogenic stimuli in order to ascertain how changes in these single cell responses lead to emerging multi-cellular behavior such as sprout formation. Here we employ single-cell microscopy to ascertain phenotypic behaviors of more than 800 human microvascular endothelial cells under various combinational angiogenic (VEGF) and angiostatic (PF4) cytokine treatments, analyzing their dynamic behavioral transitions among sessile, migratory, proliferative, and apoptotic states. We find that an endothelial cell population clusters into an identifiable set of a few distinct phenotypic state transition patterns (clusters) that is consistent across all cytokine conditions. Varying the cytokine conditions, such as VEGF and PF4 combinations here, modulates the proportion of the population following a particular pattern (referred to as phenotypic cluster weights) without altering the transition dynamics within the patterns. We then map the phenotypic cluster weights to quantified population level sprout densities using a multi-variate regression approach, and identify linear combinations of the phenotypic cluster weights that associate with greater or lesser sprout density across the various treatment conditions. VEGF-dominant cytokine combinations yielding high sprout densities are characterized by high proliferative and low apoptotic cluster weights, whereas PF4-dominant conditions yielding low sprout densities are characterized by low proliferative and high apoptotic cluster weights. Migratory cluster weights show only mild association with sprout density outcomes under the VEGF/PF4 conditions and the sprout formation characteristics explored here.National Science Foundation (U.S.) (NSF grant EFRI-0735007)National Institutes of Health (U.S.) (NIH grant R01-GM081336)National Institutes of Health (U.S.) (NIH grant R01-EB010246

Structure and Dynamics of Information Pathways in Online Media

Diffusion of information, spread of rumors and infectious diseases are all instances of stochastic processes that occur over the edges of an underlying network. Many times networks over which contagions spread are unobserved, and such networks are often dynamic and change over time. In this paper, we investigate the problem of inferring dynamic networks based on information diffusion data. We assume there is an unobserved dynamic network that changes over time, while we observe the results of a dynamic process spreading over the edges of the network. The task then is to infer the edges and the dynamics of the underlying network. We develop an on-line algorithm that relies on stochastic convex optimization to efficiently solve the dynamic network inference problem. We apply our algorithm to information diffusion among 3.3 million mainstream media and blog sites and experiment with more than 179 million different pieces of information spreading over the network in a one year period. We study the evolution of information pathways in the online media space and find interesting insights. Information pathways for general recurrent topics are more stable across time than for on-going news events. Clusters of news media sites and blogs often emerge and vanish in matter of days for on-going news events. Major social movements and events involving civil population, such as the Libyan's civil war or Syria's uprise, lead to an increased amount of information pathways among blogs as well as in the overall increase in the network centrality of blogs and social media sites.Comment: To Appear at the 6th International Conference on Web Search and Data Mining (WSDM '13

CLIC: Curriculum Learning and Imitation for object Control in non-rewarding environments

In this paper we study a new reinforcement learning setting where the environment is non-rewarding, contains several possibly related objects of various controllability, and where an apt agent Bob acts independently, with non-observable intentions. We argue that this setting defines a realistic scenario and we present a generic discrete-state discrete-action model of such environments. To learn in this environment, we propose an unsupervised reinforcement learning agent called CLIC for Curriculum Learning and Imitation for Control. CLIC learns to control individual objects in its environment, and imitates Bob's interactions with these objects. It selects objects to focus on when training and imitating by maximizing its learning progress. We show that CLIC is an effective baseline in our new setting. It can effectively observe Bob to gain control of objects faster, even if Bob is not explicitly teaching. It can also follow Bob when he acts as a mentor and provides ordered demonstrations. Finally, when Bob controls objects that the agent cannot, or in presence of a hierarchy between objects in the environment, we show that CLIC ignores non-reproducible and already mastered interactions with objects, resulting in a greater benefit from imitation