A Framework for Human Motion Strategy Identification and Analysis

The human body has many biomechanical degrees of freedom and thus multiple movement strategies can be employed to execute any given task. Automated identification and classification of these movement strategies have potential applications in various fields including sports performance research, rehabilitation, and injury prevention. For example, in the field of rehabilitation, the choice of movement strategy can impact joint loading patterns and risk of injury. The problem of identifying movement strategies is related to the problem of classifying variations in the observed motions. When differences between two movement trajectories performing the same task are large, they are considered to be different movement strategies. Conversely, when the differences between observed movements are small, they are considered to be variations of the same movement strategy. In the simplest scenario a movement strategy can represent a cluster of similar movement trajectories, but in more complicated scenarios differences in movements could also lie on a continuum. The goal of this thesis is to develop a computational framework to automatically recognize different movement strategies for performing a task and to identify what makes each strategy different. The proposed framework utilizes Gaussian Process Dynamical Models (GPDM) to convert human motion trajectories from their original high dimensional representation to a trajectory in a lower dimensional space (i.e. the latent space). The dimensionality of the latent space is determined by iteratively increasing the dimensionality until the reduction in reconstruction error between iterations becomes small. Then, the lower dimensional trajectories are clustered using a Hidden Markov Model (HMM) clustering algorithm to identify movement strategies in an unsupervised manner. Next, we introduce an HMM-based technique for detecting differences in signals between two HMM models. This technique is used to compare latent space variables between the low-dimensional trajectory models as well as differences in degrees-of-freedom (DoF) between the corresponding high-dimensional (original) trajectory models. Then, through correlating latent variable and DoF differences movement synergies are discovered. To validate the proposed framework, it was tested on 3 different datasets – a synthetic dataset, a real labeled motion capture dataset, and an unlabeled motion capture dataset. The proposed framework achieved higher classification accuracy against competing algorithms (Joint Component Vector and Kinematic Synergies) where labels were known apriori. Additionally, the proposed algorithm showed that it was able to discover strategies that were not known apriori and how the strategies differed

Key body pose detection and movement assessment of fitness performances

Motion segmentation plays an important role in human motion analysis. Understanding the intrinsic features of human activities represents a challenge for modern science. Current solutions usually involve computationally demanding processing and achieve the best results using expensive, intrusive motion capture devices. In this thesis, research has been carried out to develop a series of methods for affordable and effective human motion assessment in the context of stand-up physical exercises. The objective of the research was to tackle the needs for an autonomous system that could be deployed in nursing homes or elderly people's houses, as well as rehabilitation of high profile sport performers. Firstly, it has to be designed so that instructions on physical exercises, especially in the case of elderly people, can be delivered in an understandable way. Secondly, it has to deal with the problem that some individuals may find it difficult to keep up with the programme due to physical impediments. They may also be discouraged because the activities are not stimulating or the instructions are hard to follow. In this thesis, a series of methods for automatic assessment production, as a combination of worded feedback and motion visualisation, is presented. The methods comprise two major steps. First, a series of key body poses are identified upon a model built by a multi-class classifier from a set of frame-wise features extracted from the motion data. Second, motion alignment (or synchronisation) with a reference performance (the tutor) is established in order to produce a second assessment model. Numerical assessment, first, and textual feedback, after, are delivered to the user along with a 3D skeletal animation to enrich the assessment experience. This animation is produced after the demonstration of the expert is transformed to the current level of performance of the user, in order to help encourage them to engage with the programme. The key body pose identification stage follows a two-step approach: first, the principal components of the input motion data are calculated in order to reduce the dimensionality of the input. Then, candidates of key body poses are inferred using multi-class, supervised machine learning techniques from a set of training samples. Finally, cluster analysis is used to refine the result. Key body pose identification is guaranteed to be invariant to the repetitiveness and symmetry of the performance. Results show the effectiveness of the proposed approach by comparing it against Dynamic Time Warping and Hierarchical Aligned Cluster Analysis. The synchronisation sub-system takes advantage of the cyclic nature of the stretches that are part of the stand-up exercises subject to study in order to remove out-of-sequence identified key body poses (i.e., false positives). Two approaches are considered for performing cycle analysis: a sequential, trivial algorithm and a proposed Genetic Algorithm, with and without prior knowledge on cyclic sequence patterns. These two approaches are compared and the Genetic Algorithm with prior knowledge shows a lower rate of false positives, but also a higher false negative rate. The GAs are also evaluated with randomly generated periodic string sequences. The automatic assessment follows a similar approach to that of key body pose identification. A multi-class, multi-target machine learning classifier is trained with features extracted from previous motion alignment. The inferred numerical assessment levels (one per identified key body pose and involved body joint) are translated into human-understandable language via a highly-customisable, context-free grammar. Finally, visual feedback is produced in the form of a synchronised skeletal animation of both the user's performance and the tutor's. If the user's performance is well below a standard then an affine offset transformation of the skeletal motion data series to an in-between performance is performed, in order to prevent dis-encouragement from the user and still provide a reference for improvement. At the end of this thesis, a study of the limitations of the methods in real circumstances is explored. Issues like the gimbal lock in the angular motion data, lack of accuracy of the motion capture system and the escalation of the training set are discussed. Finally, some conclusions are drawn and future work is discussed

Reusing a robot's behavioral mechanisms to model and manipulate human mental states

Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2010.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 125-129).In a task domain characterized by physical actions and where information has value, competing teams gain advantage by spying on and deceiving an opposing team while cooperating teammates can help the team by secretly communicating new information. For a robot to thrive in this environment it must be able to perform actions in a manner to deceive opposing agents as well as to be able to secretly communicate with friendly agents. It must further be able to extract information from observing the actions of other agents. The goal of this research is to expand on current human robot interaction by creating a robot that can operate in the above scenario. To enable these behaviors, an architecture is created which provides the robot with mechanisms to work with hidden human mental states. The robot attempts to infer these hidden states from observable factors and use them to better understand and predict behavior. It also takes steps to alter them in order to change the future behavior of the other agent. It utilizes the knowledge that the human is performing analogous inferences about the robot's own internal states to predict the effect of its actions on the human's knowledge and perceptions of the robot. The research focuses on the implicit communication that is made possible by two embodied agents interacting in a shared space through nonverbal interaction. While the processes used by a robot differ significantly from the cognitive mechanisms employed by humans, each face the similar challenge of completing the loop from sensing to acting. This architecture employs a self-as-simulator strategy, reusing the robot's behavioral mechanisms to model aspects of the human's mental states. This reuse allows the robot to model human actions and the mental states behind them using the grammar of its own representations and actions.by Jesse Vail Gray.Ph.D

Motion capture based motion analysis and motion synthesis for human-like character animation.

Motion capture technology is recognised as a standard tool in the computer animation pipeline. It provides detailed movement for animators; however, it also introduces problems and brings concerns for creating realistic and convincing motion for character animation. In this thesis, the post-processing techniques are investigated that result in realistic motion generation. Anumber of techniques are introduced that are able to improve the quality of generated motion from motion capture data, especially when integrating motion transitions from different motion clips. The presented motion data reconstruction technique is able to build convincing realistic transitions from existing motion database, and overcome the inconsistencies introduced by traditional motion blending techniques. It also provides a method for animators to re-use motion data more efficiently. Along with the development of motion data transition reconstruction, the motion capture data mapping technique was investigated for skeletal movement estimation. The per-frame based method provides animators with a real-time and accurate solution for a key post-processing technique. Although motion capture systems capture physically-based motion for character animation, no physical information is included in the motion capture data file. Using the knowledge of biomechanics and robotics, the relevant information for the captured performer are able to be abstracted and a mathematical-physical model are able to be constructed; such information is then applied for physics-based motion data correction whenever the motion data is edited