Analyzing human gait and posture by combining feature selection and kernel methods

This paper evaluates a set of computational algorithms for the automatic estimation of human postures and gait properties from signals provided by an inertial body sensor. The use of a single sensor device imposes limitations for the automatic estimation of relevant properties, like step length and gait velocity, as well as for the detection of standard postures like sitting or standing. Moreover, the exact location and orientation of the sensor is also a common restriction that is relaxed in this study. Based on accelerations provided by a sensor, known as the `9 2', three approaches are presented extracting kinematic information from the user motion and posture. Firstly, a two-phases procedure implementing feature extraction and Support Vector Machine based classi cation for daily living activity monitoring is presented. Secondly, Support Vector Regression is applied on heuristically extracted features for the automatic computation of spatiotemporal properties during gait. Finally, sensor information is interpreted as an observation of a particular trajectory of the human gait dynamical system, from which a reconstruction space is obtained, and then transformed using standard principal components analysis, nally Support Vector Regression is used for prediction. Daily living Activities are detected and spatiotemporal parameters of human gait are estimated using methods sharing a common structure based on feature extraction and kernel methods. The approaches presented are susceptible to be used for medical purposes.Peer ReviewedPreprin

Use of Machine Learning and Wearable Sensors to Predict Energetics and Kinematics of Cutting Maneuvers

Changes of directions and cutting maneuvers, including 180-degree turns, are common locomotor actions in team sports, implying high mechanical load. While the mechanics and neurophysiology of turns have been extensively studied in laboratory conditions, modern inertial measurement units allow us to monitor athletes directly on the field. In this study, we applied four supervised machine learning techniques (linear regression, support vector regression/machine, boosted decision trees and artificial neural networks) to predict turn direction, speed (before/after turn) and the related positive/negative mechanical work. Reference values were computed using an optical motion capture system. We collected data from 13 elite female soccer players performing a shuttle run test, wearing a six-axes inertial sensor at the pelvis level. A set of 18 features (predictors) were obtained from accelerometers, gyroscopes and barometer readings. Turn direction classification returned good results (accuracy > 98.4%) with all methods. Support vector regression and neural networks obtained the best performance in the estimation of positive/negative mechanical work (coefficient of determination R-2 = 0.42-0.43, mean absolute error = 1.14-1.41 J) and running speed before/after the turns (R-2 = 0.66-0.69, mean absolute error = 0.15-018 m/s). Although models can be extended to different angles, we showed that meaningful information on turn kinematics and energetics can be obtained from inertial units with a data-driven approach

An Advanced Adaptive Control of Lower Limb Rehabilitation Robot

Rehabilitation robots play an important role in the rehabilitation field, and effective human-robot interaction contributes to promoting the development of the rehabilitation robots. Though many studies about the human-robot interaction have been carried out, there are still several limitations in the flexibility and stability of the control system. Therefore, we proposed an advanced adaptive control method for lower limb rehabilitation robot. The method was devised with a dual closed loop control strategy based on the surface electromyography (sEMG) and plantar pressure to improve the robustness of the adaptive control for the rehabilitation robots. First, in the outer loop control, an advanced variable impedance controller based on the sEMG and plantar pressure was designed to correct robot's reference trajectory. Then, in the inner loop control, a sliding mode iterative learning controller (SMILC) based on the variable boundary saturation function was designed to achieve the tracking of the reference trajectory. The experiment results showed that, in the designed dual closed loop control strategy, a variable impedance controller can effectively reduce trajectory tracking errors and adaptively modify the reference trajectory synchronizing with the motion intention of patients; the designed sliding mode iterative learning controller can effectively reduce chattering in sliding mode control and excellently achieve the tracking of rehabilitation robot's reference trajectory. This study can improve the performance of the human-robot interaction of the rehabilitation robot system, and expand the application to the rehabilitation field

Dynamic estimation of human energy expenditure with wearable sensors

&nbsp;This study investigated energy expenditure estimation using inertial measurement units, electromyography and O2 Gas Mask sensors whilst undergoing motion. Further investigations included the precision measurement of IMUs in order to achieve extremely accurate energy estimation calculations using an experimental Dual-Kalman Filter and limb length estimation using entropy based methods .<br /

Stability and Performance Improvement in Haptic Human-Robot Interaction

The goal of this research is to develop theories, methods, and tools to understand the mechanisms of neuromotor adaptation in human-robot physical interaction, in order to improve the stability and performance of the interaction. Human power-assisting systems (e.g., powered lifting devices that aid human operators in manipulating heavy or bulky loads) require physical contact between the operator and machine, creating a coupled dynamic system. This dynamic coupling has been shown to introduce inherent instabilities and performance degradation due to a change in human stiffness; when instability is encountered, a human operator often attempts to control the oscillation by stiffening their arm, which leads to a stiffer system with more instability. Robot co-worker controllers must account for this issue. In this work we set out to 1) understand the association between neuromuscular adaptations and system performance limits, 2) develop probabilistic methods to classify and predict the transition of operator’s cognitive and physical states from physiological measures, and 3) integrate this knowledge into a structure of shared human-robot control. We developed a model of the human operator endpoint stiffness, characterized at the musculoskeletal level, that can account for deliberate stiffness increase at the endpoint through the incorporation of muscle coactivation. We also developed a switching admittance control approach which can account for changes in the operator’s muscle coactivation and is able to generate cognitive states in an unsupervised manner, given a relevant training dataset. Finally, a novel variable admittance control approach, which significantly reduces grasp contact instability commonly encountered in fixed admittance control settings, was developed, analytically derived, and provides solutions for both constant mass and variable mass parameter cases.Ph.D

Mechanisms of motor learning: by humans, for robots

Whenever we perform a movement and interact with objects in our environment, our central nervous system (CNS) adapts and controls the redundant system of muscles actuating our limbs to produce suitable forces and impedance for the interaction. As modern robots are increasingly used to interact with objects, humans and other robots, they too require to continuously adapt the interaction forces and impedance to the situation. This thesis investigated the motor mechanisms in humans through a series of technical developments and experiments, and utilized the result to implement biomimetic motor behaviours on a robot. Original tools were first developed, which enabled two novel motor imaging experiments using functional magnetic resonance imaging (fMRI). The first experiment investigated the neural correlates of force and impedance control to understand the control structure employed by the human brain. The second experiment developed a regressor free technique to detect dynamic changes in brain activations during learning, and applied this technique to investigate changes in neural activity during adaptation to force fields and visuomotor rotations. In parallel, a psychophysical experiment investigated motor optimization in humans in a task characterized by multiple error-effort optima. Finally a computational model derived from some of these results was implemented to exhibit human like control and adaptation of force, impedance and movement trajectory in a robot

Stochastic optimal control with learned dynamics models

The motor control of anthropomorphic robotic systems is a challenging computational task mainly because of the high levels of redundancies such systems exhibit. Optimality principles provide a general strategy to resolve such redundancies in a task driven fashion. In particular closed loop optimisation, i.e., optimal feedback control (OFC), has served as a successful motor control model as it unifies important concepts such as costs, noise, sensory feedback and internal models into a coherent mathematical framework. Realising OFC on realistic anthropomorphic systems however is non-trivial: Firstly, such systems have typically large dimensionality and nonlinear dynamics, in which case the optimisation problem becomes computationally intractable. Approximative methods, like the iterative linear quadratic gaussian (ILQG), have been proposed to avoid this, however the transfer of solutions from idealised simulations to real hardware systems has proved to be challenging. Secondly, OFC relies on an accurate description of the system dynamics, which for many realistic control systems may be unknown, difficult to estimate, or subject to frequent systematic changes. Thirdly, many (especially biologically inspired) systems suffer from significant state or control dependent sources of noise, which are difficult to model in a generally valid fashion. This thesis addresses these issues with the aim to realise efficient OFC for anthropomorphic manipulators. First we investigate the implementation of OFC laws on anthropomorphic hardware. Using ILQG we optimally control a high-dimensional anthropomorphic manipulator without having to specify an explicit inverse kinematics, inverse dynamics or feedback control law. We achieve this by introducing a novel cost function that accounts for the physical constraints of the robot and a dynamics formulation that resolves discontinuities in the dynamics. The experimental hardware results reveal the benefits of OFC over traditional (open loop) optimal controllers in terms of energy efficiency and compliance, properties that are crucial for the control of modern anthropomorphic manipulators. We then propose a new framework of OFC with learned dynamics (OFC-LD) that, unlike classic approaches, does not rely on analytic dynamics functions but rather updates the internal dynamics model continuously from sensorimotor plant feedback. We demonstrate how this approach can compensate for unknown dynamics and for complex dynamic perturbations in an online fashion. A specific advantage of a learned dynamics model is that it contains the stochastic information (i.e., noise) from the plant data, which corresponds to the uncertainty in the system. Consequently one can exploit this information within OFC-LD in order to produce control laws that minimise the uncertainty in the system. In the domain of antagonistically actuated systems this approach leads to improved motor performance, which is achieved by co-contracting antagonistic actuators in order to reduce the negative effects of the noise. Most importantly the shape and source of the noise is unknown a priory and is solely learned from plant data. The model is successfully tested on an antagonistic series elastic actuator (SEA) that we have built for this purpose. The proposed OFC-LD model is not only applicable to robotic systems but also proves to be very useful in the modelling of biological motor control phenomena and we show how our model can be used to predict a wide range of human impedance control patterns during both, stationary and adaptation tasks

A Continuous Grasp Representation for the Imitation Learning of Grasps on Humanoid Robots

Models and methods are presented which enable a humanoid robot to learn reusable, adaptive grasping skills. Mechanisms and principles in human grasp behavior are studied. The findings are used to develop a grasp representation capable of retaining specific motion characteristics and of adapting to different objects and tasks. Based on the representation a framework is proposed which enables the robot to observe human grasping, learn grasp representations, and infer executable grasping actions

Generative Models for Learning Robot Manipulation Skills from Humans

A long standing goal in artificial intelligence is to make robots seamlessly interact with humans in performing everyday manipulation skills. Learning from demonstrations or imitation learning provides a promising route to bridge this gap. In contrast to direct trajectory learning from demonstrations, many problems arise in interactive robotic applications that require higher contextual level understanding of the environment. This requires learning invariant mappings in the demonstrations that can generalize across different environmental situations such as size, position, orientation of objects, viewpoint of the observer, etc. In this thesis, we address this challenge by encapsulating invariant patterns in the demonstrations using probabilistic learning models for acquiring dexterous manipulation skills. We learn the joint probability density function of the demonstrations with a hidden semi-Markov model, and smoothly follow the generated sequence of states with a linear quadratic tracking controller. The model exploits the invariant segments (also termed as sub-goals, options or actions) in the demonstrations and adapts the movement in accordance with the external environmental situations such as size, position and orientation of the objects in the environment using a task-parameterized formulation. We incorporate high-dimensional sensory data for skill acquisition by parsimoniously representing the demonstrations using statistical subspace clustering methods and exploit the coordination patterns in latent space. To adapt the models on the fly and/or teach new manipulation skills online with the streaming data, we formulate a non-parametric scalable online sequence clustering algorithm with Bayesian non-parametric mixture models to avoid the model selection problem while ensuring tractability under small variance asymptotics. We exploit the developed generative models to perform manipulation skills with remotely operated vehicles over satellite communication in the presence of communication delays and limited bandwidth. A set of task-parameterized generative models are learned from the demonstrations of different manipulation skills provided by the teleoperator. The model captures the intention of teleoperator on one hand and provides assistance in performing remote manipulation tasks on the other hand under varying environmental situations. The assistance is formulated under time-independent shared control, where the model continuously corrects the remote arm movement based on the current state of the teleoperator; and/or time-dependent autonomous control, where the model synthesizes the movement of the remote arm for autonomous skill execution. Using the proposed methodology with the two-armed Baxter robot as a mock-up for semi-autonomous teleoperation, we are able to learn manipulation skills such as opening a valve, pick-and-place an object by obstacle avoidance, hot-stabbing (a specialized underwater task akin to peg-in-a-hole task), screw-driver target snapping, and tracking a carabiner in as few as 4 - 8 demonstrations. Our study shows that the proposed manipulation assistance formulations improve the performance of the teleoperator by reducing the task errors and the execution time, while catering for the environmental differences in performing remote manipulation tasks with limited bandwidth and communication delays

Tensor-based regression models and applications

Tableau d’honneur de la Faculté des études supérieures et postdoctorales, 2017-2018Avec l’avancement des technologies modernes, les tenseurs d’ordre élevé sont assez répandus et abondent dans un large éventail d’applications telles que la neuroscience informatique, la vision par ordinateur, le traitement du signal et ainsi de suite. La principale raison pour laquelle les méthodes de régression classiques ne parviennent pas à traiter de façon appropriée des tenseurs d’ordre élevé est due au fait que ces données contiennent des informations structurelles multi-voies qui ne peuvent pas être capturées directement par les modèles conventionnels de régression vectorielle ou matricielle. En outre, la très grande dimensionnalité de l’entrée tensorielle produit une énorme quantité de paramètres, ce qui rompt les garanties théoriques des approches de régression classique. De plus, les modèles classiques de régression se sont avérés limités en termes de difficulté d’interprétation, de sensibilité au bruit et d’absence d’unicité. Pour faire face à ces défis, nous étudions une nouvelle classe de modèles de régression, appelés modèles de régression tensor-variable, où les prédicteurs indépendants et (ou) les réponses dépendantes prennent la forme de représentations tensorielles d’ordre élevé. Nous les appliquons également dans de nombreuses applications du monde réel pour vérifier leur efficacité et leur efficacité.With the advancement of modern technologies, high-order tensors are quite widespread and abound in a broad range of applications such as computational neuroscience, computer vision, signal processing and so on. The primary reason that classical regression methods fail to appropriately handle high-order tensors is due to the fact that those data contain multiway structural information which cannot be directly captured by the conventional vector-based or matrix-based regression models, causing substantial information loss during the regression. Furthermore, the ultrahigh dimensionality of tensorial input produces huge amount of parameters, which breaks the theoretical guarantees of classical regression approaches. Additionally, the classical regression models have also been shown to be limited in terms of difficulty of interpretation, sensitivity to noise and absence of uniqueness. To deal with these challenges, we investigate a novel class of regression models, called tensorvariate regression models, where the independent predictors and (or) dependent responses take the form of high-order tensorial representations. We also apply them in numerous real-world applications to verify their efficiency and effectiveness. Concretely, we first introduce hierarchical Tucker tensor regression, a generalized linear tensor regression model that is able to handle potentially much higher order tensor input. Then, we work on online local Gaussian process for tensor-variate regression, an efficient nonlinear GPbased approach that can process large data sets at constant time in a sequential way. Next, we present a computationally efficient online tensor regression algorithm with general tensorial input and output, called incremental higher-order partial least squares, for the setting of infinite time-dependent tensor streams. Thereafter, we propose a super-fast sequential tensor regression framework for general tensor sequences, namely recursive higher-order partial least squares, which addresses issues of limited storage space and fast processing time allowed by dynamic environments. Finally, we introduce kernel-based multiblock tensor partial least squares, a new generalized nonlinear framework that is capable of predicting a set of tensor blocks by merging a set of tensor blocks from different sources with a boosted predictive power