Bio-inspired Dynamic 3D Discriminative Skeletal Features for Human Action Recognition

Abstract

Over the last few years, with the immense popularity of the Kinect, there has been renewed interest in developing methods for human gesture and action recognition from 3D data. A number of approaches have been proposed that ex-tract representative features from 3D depth data, a recon-structed 3D surface mesh or more commonly from the re-covered estimate of the human skeleton. Recent advances in neuroscience have discovered a neural encoding of static 3D shapes in primate infero-temporal cortex that can be represented as a hierarchy of medial axis and surface fea-tures. We hypothesize a similar neural encoding might also exist for 3D shapes in motion and propose a hierarchy of dynamic medial axis structures at several spatio-temporal scales that can be modeled using a set of Linear Dynami-cal Systems (LDSs). We then propose novel discriminative metrics for comparing these sets of LDSs for the task of hu-man activity recognition. Combined with simple classifica-tion frameworks, our proposed features and corresponding hierarchical dynamical models provide the highest human activity recognition rates as compared to state-of-the-art methods on several skeletal datasets. 1