Parametric Hidden Markov Models for Recognition and Synthesis of Movements

In humanoid robotics, the recognition and synthesis of parametric movements plays an extraordinary role for robot human interaction. Such a parametric movement is a movement of a particular type (semantic), for example, similar pointing movements performed at different table-top positions. For understanding the whole meaning of a movement of a human, the recognition of its type, likewise its parameterization are important. Only both together convey the whole meaning. Vice versa, for mimicry, the synthesis of movements for the motor control of a robot needs to be parameterized, e.g., by the relative position a grasping action is performed at. For both cases, synthesis and recognition, only parametric approaches are meaningful as it is not feasible to store, or acquire all possible trajectories. In this paper, we use hidden Markov models (HMMs) extended in an exemplar-based parametric way (PHMM) to represent parametric movements. As HMMs are generative, they are well suited for synthesis as well as for recognition. Synthesis and recognition are carried out through interpolation of exemplar movements to generalize over the parameterization of a movement class. In the evaluation of the approach we concentrate on a systematical validation for two parametric movements, grasping and pointing. Even though the movements are very similar in appearance our approach is able to distinguish the two movement types reasonable well. In further experiments, we show the applicability for online recognition based on very noisy 3D tracking data. The use of a parametric representation of movements is shown in a robot demo, where a robot removes objects from a table as demonstrated by an advisor. The synthesis for motor control is performed for arbitrary table-top positions

Human motion analysis and simulation tools

Análise de movimento é actualmente um tópico de pesquisa bastante activo nas áreas da Visão por Computador,Computação Gráfica e Biomecânica, devido à sua aplicabilidade num vasto espectro de aplicações em diversasáreas. Com este trabalho pretendemos apresentar um detalhado, abrangente e atualizado estudo sobre aplica-ções de análise e/ou de simulação de movimento, que têm sido desenvolvidas tanto pela comunidade científicacomo por entidades comerciais. O principal contributo deste estudo, além da listagem abrangente de ferramentasde análise de movimento, é a apresentação de um esquema eficaz para classificar e comparar ferramentasde simulação e de análise de movimento.Motion analysis is currently an active research topic in Computational Vision, Computer Graphics, Machine Learning and Biomechanics mainly due to its applicability into a wide spectrum of relevant applications in many areas. This work intends to present a detailed, broad and up to date survey on motion and/or simulation analysis software packages that have been developed both by the scientific community and commercial entities, to be used in the field of biomechanics. The main contribution of this study, beyond the comprehensive listing of motion analysis tools, is the presentation of an effective framework to classify and compare motion simulation and analysis tools

The Meaning of Action:a review on action recognition and mapping

In this paper, we analyze the different approaches taken to date within the computer vision, robotics and artificial intelligence communities for the representation, recognition, synthesis and understanding of action. We deal with action at different levels of complexity and provide the reader with the necessary related literature references. We put the literature references further into context and outline a possible interpretation of action by taking into account the different aspects of action recognition, action synthesis and task-level planning

Machine Learning Approaches to Center-of-mass Estimation from Noisy Human Motion Data

The focus of this research is to estimate Center Of Mass (COM) from noisy motion data. COM is a 3D point in the human body around which the mass of the whole body is equally distributed in each direction, and it plays an important role in many biomechanical studies of human motion, such as gait stability assessment. Traditionally, COM is computed using the Dempster's technique where the total COM is the sum of the weighted segmental COMs. Computation of COM normally requires expensive optical, mechanical and electro-magnetic motion capture systems (MOCAP). Instead of high precision MOCAP systems, we can use low-cost sensors such as inertial motion sensors for efficient motion acquisition to compute COM. However, sensor-based motion acquisition could be noisy due to various ambient interference conditions and may be incomplete due to a limited number of sensors used. As a result, direct computation of COM from noisy motion data could be unreliable and even unusable in practice. In this research we have proposed two machine approaches to address this problem, i.e., manifold mapping and Gaussian Process Regression (GPR). First, we have designed a torus manifold which is a low-dimensional space to represent complex motion kinematics via two variables, i.e., the gait and the pose, representing different walking styles and different stages in a walking cycle, respectively. This torus manifold is shared by motion data (MOCAP) and the corresponding COM trajectories and provides with continuous space to extrapolate unknown motion along its COM trajectory. Moreover, given a noisy motion sequence, the torus manifold can be used to play a filtering role to denoise the motion data as well as a bridge to map the filtered motion data to the corresponding COM sequence. On the other hand, GPR does not account motion kinematics explicitly, and it directly approximates a non-linear mapping function between the input space (motion data) to the output space (COM data) where the covariance structure learned from noiseless motion data plays an important role for COM prediction. Our experiment has shown that GPR works better than the torus manifold for COM estimation from noiseless motion data. However, the performance of GPR degrades as the noise level increases in the motion data, largely due to the fact that its dependence on the covariance structure learned from the noiseless training data does not match that of the noisy motion data. In other words, unlike the torus manifold-based method, there is no filtering effect from GRP which makes it less accurate to estimate COM under noisy motion data. Still, both machine learning techniques have shown significant advantage over the method of direct computation of COM from noisy motion data.School of Electrical & Computer Engineerin

A virtual environment for the modelling, simulation and manufacturing of orthopaedic devices

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The objective of this work is to investigate whether the game physics based modelling is accurate enough to be used in modelling the motion of the human body, in particular musculoskeletal motion. Hitherto, the implementation of game physics in the medical field focused only on anatomical representation for education and training purposes. Introducing gaming platforms and physics engines into orthopaedics applications will help to overcome several difficulties encountered in the modelling of articular joints. Implementing a physics engine (PhysX), which is mainly designed for video games, handles intensive computations in optimized ways at an interactive speed. In this study, the capabilities of the physics engine (PhysX) and gaming platform for modelling and simulating articular joints are evaluated. First, a preliminary validation is carried out for mechanical systems with analytical solutions, before constructing the musculoskeletal model to evaluate the consistency of gaming platforms. The developed musculoskeletal model deals with the human joint as an unconstrained system with 6 DOF which is not available with other joint modeller. The model articulation is driven by contact surfaces and the stiffness of surrounding tissues. A number of contributions, such as contact modelling and muscle wrapping, have been made in this research to overcome some existing challenges in joint modelling. Using muscle segmentation, the proposed technique effectively handles the problem of muscle wrapping, a major concern for many; thus the shortest path and line of action are no longer problematic. Collision behaviour has also shown a stable response for colliding as well as resting objects, provided that it is based on the principles of surface properties and the conservation of linear and angular momentums. The precision of collision detection and response are within an acceptable tolerance controllable by varying the mesh density. An image based analysis system is developed in this thesis, mainly in order to validate the proposed physics based modelling solution. This minimally invasive method is based on the analysis of marker positions located at bony positions with minimal skin movement. The image based system overcomes several challenges associated with the currently existing methods, such as inaccuracy, complication, impracticability and cost. The analysis part of this research has considered the elbow joint as a case study to investigate and validate the proposed physics based model. Beside the interactive 3D simulation, the obtained results are validated by comparing them with the image based system developed within the current research to investigate joint kinematics and laxity and also with published material, MJM and results from experiments performed at the Brunel Orthopaedic Research and Learning Centre. The proposed modelling shows the advantageous speed, reliability and flexibility of the proposed model. It is shown that the gaming platform and physics engine provide a viable solution to human musculoskeletal modelling. Finally, this thesis considers an extended implementation of the proposed platform for testing and assessing the design of custom-made implants, to enhance joint performance. The developed simulation software is expected to give indicative results as well as testing different types of prosthetic implant. Design parameterization and sensitivity analysis for geometrical features are discussed. Thus, an integrated environment is proposed to link the real-time simulation software with a manufacturing environment so as to assist the production of patient specific implants by rapid manufacturing

A pervasive body sensor network for monitoring post-operative recovery

Over the past decade, miniaturisation and cost reduction brought about by the semiconductor industry has led to computers smaller in size than a pin head, powerful enough to carry out the processing required, and affordable enough to be disposable. Similar technological advances in wireless communication, sensor design, and energy storage have resulted in the development of wireless “Body Sensor Network (BSN) platforms comprising of tiny integrated micro sensors with onboard processing and wireless data transfer capability, offering the prospect of pervasive and continuous home health monitoring. In surgery, the reduced trauma of minimally invasive interventions combined with initiatives to reduce length of hospital stay and a socioeconomic drive to reduce hospitalisation costs, have all resulted in a trend towards earlier discharge from hospital. There is now a real need for objective, pervasive, and continuous post-operative home recovery monitoring systems. Surgical recovery is a multi-faceted and dynamic process involving biological, physiological, functional, and psychological components. Functional recovery (physical independence, activities of daily living, and mobility) is recognised as a good global indicator of a patient’s post-operative course, but has traditionally been difficult to objectively quantify. This thesis outlines the development of a pervasive wireless BSN system to objectively monitor the functional recovery of post-operative patients at home. Biomechanical markers were identified as surrogate measures for activities of daily living and mobility impairment, and an ear-worn activity recognition (e-AR) sensor containing a three-axis accelerometer and a pulse oximeter was used to collect this data. A simulated home environment was created to test a Bayesian classifier framework with multivariate Gaussians to model activity classes. A real-time activity index was used to provide information on the intensity of activity being performed. Mobility impairment was simulated with bracing systems and a multiresolution wavelet analysis and margin-based feature selection framework was used to detect impaired mobility. The e-AR sensor was tested in a home environment before its clinical use in monitoring post-operative home recovery of real patients who have undergone surgery. Such a system may eventually form part of an objective pervasive home recovery monitoring system tailored to the needs of today’s post-operative patient.Open acces

Quantitative Estimation of Movement Progress during Rehabilitation after Knee/Hip Replacement Surgery

Mobility improvement for patients is one of the primary concerns of physiotherapy rehabilitation. In a typical physiotherapy session, the patient is instructed to perform multiple exercises, based on a specific regimen recommended by the physiotherapist for each patient. The physiotherapist then evaluates the patient's progress based on his or her performance during the exercises. Providing the physiotherapist and the patient with a quantified and objective measure of progress, based on both individual exercises and the exercise set, can be beneficial for monitoring the patient's performance. The quantified measure can also be beneficial when the physiotherapist is not available, e.g., crowded gym or rehabilitation at home. In this thesis, two approaches are introduced for quantifying patient performance. One approach describes the movement timeseries by statistical measures and the other by a stochastic model. Both approaches formulate a distance between patient data and the healthy population as the measure of performance. Distance measures are defined to capture the performance of one repetition of an exercise or multiple repetitions of the same exercise. To capture patient progress across multiple exercises, a quality measure and overall score are formulated based on the distance measures and are used to quantify the overall performance for each session. The proposed approaches are compared to several existing approaches, including sample distribution approaches (two sample kernel), classifier-based approaches (Naive Bayes, Support Vector Machines, and Kullback-Leibler Divergence), and dynamical movement primitives. In their original formulation, existing approaches are not capable of estimating measures of performance for multiple exercises. Therefore, the measures of performance for multiple repetitions of the same exercise are estimated using the existing approaches, while the formulation proposed in this thesis is used to estimate the overall performance for multiple exercises in one session. The effects of different variabilities in human motion on the performance of the proposed approaches and the comparison approaches are investigated with both synthetic and patient data. The patient data consists of rehabilitation data recorded from patients recovering from knee or hip replacement surgery, the associated exercise regimen and physiotherapist evaluations of progress. The methods are evaluated quantitatively based on correlation between methods, correlation with exercise regimen difficulty, and qualitatively based on the patients' medical charts. The proposed approaches are capable of capturing the trend of progress for the synthetic dataset and are superior to the existing approaches in presence of multiple sources of variability. For patient data, the proposed approaches correlate moderately with the score obtained from the exercise regimen, and qualitatively correspond with the patients' medical charts. The results indicate that the quantified measures of progress obtained from the proposed approaches are promising tools for supporting physiotherapy practice through monitoring patient progress.1 yea