Feedback MPC for Torque-Controlled Legged Robots

The computational power of mobile robots is currently insufficient to achieve torque level whole-body Model Predictive Control (MPC) at the update rates required for complex dynamic systems such as legged robots. This problem is commonly circumvented by using a fast tracking controller to compensate for model errors between updates. In this work, we show that the feedback policy from a Differential Dynamic Programming (DDP) based MPC algorithm is a viable alternative to bridge the gap between the low MPC update rate and the actuation command rate. We propose to augment the DDP approach with a relaxed barrier function to address inequality constraints arising from the friction cone. A frequency-dependent cost function is used to reduce the sensitivity to high-frequency model errors and actuator bandwidth limits. We demonstrate that our approach can find stable locomotion policies for the torque-controlled quadruped, ANYmal, both in simulation and on hardware.Comment: Paper accepted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019

Doctor of Philosophy

dissertationMotion capture has applications in many fields. A need has arisen for motion capture systems that are low-cost, mobile, and intuitive. An attitude heading reference system (AHRS) calculates the global orientation of a rigid body by synthesizing the output from an array of sensors. A complete motion capture system utilizing gyroscopes, accelerometers, and magnetometers attached to the main body segments of a human is proposed. This is accomplished by providind a low-cost calibration procedure for micro electro-mechanical system (MEMS) gyroscopes, accelerometers, and magnetometers in order to create a custom AHRS unit. The accuracy of reproducing global orientations using these AHRS units is analyzed for individual modules as well as redundant groups of AHRS nodes for increased accuracy. In order to make the system intuitive, a localization procedure for finding the locations of all AHRS units attached to the body is proposed. Sensors were successfully calibrated to an accuracy sufficient for AHRS development. The accuracy of the AHRS units was verified and led to a functioning motion capture system. The localization procedure was verified with volunteer subjects and successfully finds the location of all attached AHRS units

Recent Advances in Motion Analysis

The advances in the technology and methodology for human movement capture and analysis over the last decade have been remarkable. Besides acknowledged approaches for kinematic, dynamic, and electromyographic (EMG) analysis carried out in the laboratory, more recently developed devices, such as wearables, inertial measurement units, ambient sensors, and cameras or depth sensors, have been adopted on a wide scale. Furthermore, computational intelligence (CI) methods, such as artificial neural networks, have recently emerged as promising tools for the development and application of intelligent systems in motion analysis. Thus, the synergy of classic instrumentation and novel smart devices and techniques has created unique capabilities in the continuous monitoring of motor behaviors in different fields, such as clinics, sports, and ergonomics. However, real-time sensing, signal processing, human activity recognition, and characterization and interpretation of motion metrics and behaviors from sensor data still representing a challenging problem not only in laboratories but also at home and in the community. This book addresses open research issues related to the improvement of classic approaches and the development of novel technologies and techniques in the domain of motion analysis in all the various fields of application

Methodische Ansätze zur Analyse biomechanischer Zeitreihendaten

Die vorliegende Dissertation beschäftigt sich mit der Frage inwiefern der Einsatz von Zeitreihenmethoden - also Methoden, die den kontinuierlichen Charakter biomechanischer Zeitreihendaten berücksichtigen - einen aus methodischer Sicht günstigeren Auswertungsansatz darstellt als herkömmliche, diskrete Methoden. Damit liefert die Arbeit einen wichtigen Beitrag zur Überwindung der postulierten Konfidenzkrise biomechanisch-bewegungswissenschaftlicher Forschung, nach der sich die Ergebnisse vieler Studien aufgrund u. a. methodischer Defizite nicht oder nur eingeschränkt reproduzieren lassen. Dabei konnte in drei Teilstudien für verschiedene methodische Bereiche (Inferenzstatistik, Klassifikation, Trenderkennung) gezeigt werden, dass der Einsatz von Zeitreihenmethoden bekannte Probleme diskreter Methoden erfolgreich addressieren kann. Die vorgestellte Arbeit regt über das spezifische Thema hinaus zu einem kritischeren Umgang mit methodischen Aspekten an und zeigt mögliche Lösungsansätze auf

Human Activity Recognition and Control of Wearable Robots

abstract: Wearable robotics has gained huge popularity in recent years due to its wide applications in rehabilitation, military, and industrial fields. The weakness of the skeletal muscles in the aging population and neurological injuries such as stroke and spinal cord injuries seriously limit the abilities of these individuals to perform daily activities. Therefore, there is an increasing attention in the development of wearable robots to assist the elderly and patients with disabilities for motion assistance and rehabilitation. In military and industrial sectors, wearable robots can increase the productivity of workers and soldiers. It is important for the wearable robots to maintain smooth interaction with the user while evolving in complex environments with minimum effort from the user. Therefore, the recognition of the user's activities such as walking or jogging in real time becomes essential to provide appropriate assistance based on the activity. This dissertation proposes two real-time human activity recognition algorithms intelligent fuzzy inference (IFI) algorithm and Amplitude omega ($A \omega$) algorithm to identify the human activities, i.e., stationary and locomotion activities. The IFI algorithm uses knee angle and ground contact forces (GCFs) measurements from four inertial measurement units (IMUs) and a pair of smart shoes. Whereas, the $A \omega$ algorithm is based on thigh angle measurements from a single IMU. This dissertation also attempts to address the problem of online tuning of virtual impedance for an assistive robot based on real-time gait and activity measurement data to personalize the assistance for different users. An automatic impedance tuning (AIT) approach is presented for a knee assistive device (KAD) in which the IFI algorithm is used for real-time activity measurements. This dissertation also proposes an adaptive oscillator method known as amplitude omega adaptive oscillator ($A\omega AO$) method for HeSA (hip exoskeleton for superior augmentation) to provide bilateral hip assistance during human locomotion activities. The $A \omega$ algorithm is integrated into the adaptive oscillator method to make the approach robust for different locomotion activities. Experiments are performed on healthy subjects to validate the efficacy of the human activities recognition algorithms and control strategies proposed in this dissertation. Both the activity recognition algorithms exhibited higher classification accuracy with less update time. The results of AIT demonstrated that the KAD assistive torque was smoother and EMG signal of Vastus Medialis is reduced, compared to constant impedance and finite state machine approaches. The $A\omega AO$ method showed real-time learning of the locomotion activities signals for three healthy subjects while wearing HeSA. To understand the influence of the assistive devices on the inherent dynamic gait stability of the human, stability analysis is performed. For this, the stability metrics derived from dynamical systems theory are used to evaluate unilateral knee assistance applied to the healthy participants.Dissertation/ThesisDoctoral Dissertation Aerospace Engineering 201

Methods and good practice guidelines for human joint kinematics estimation through magnetic and inertial wearable sensors

According to the World Health Organization, the ability to move is recognized as a key factor for the human well-being. From the wearable Magnetic and Inertial Measurement Units (MIMUs) signals it is possible to extract several digital mobility outcomes including the joint kinematics. To this end, it is first required to estimate the orientation of the MIMUs by means of a sensor fusion algorithm (SFA). After that, the relative orientation is computed and then decomposed to obtain the joint angles. However, the MIMUs do not provide a direct output of the physical quantity of interest which can be only determined after an ad hoc processing of their signals. It follows that the joint angle accuracy mostly depends on multiple factors. The first one is the magnitude of the MIMU measurements errors and up to date there is still a lack of methods for their characterization. A second crucial factor is the choice of the SFA to use. Despite the abundance of formulations in the literature, no-well established conclusions about their accuracy have been reached yet. The last factor is the biomechanical model used to compute the joint angles. In this context, unconstrained methods offer a simple way to decompose the relative orientation using the Euler angles but suffer from the inherent issues related to the SFA. In contrast, constrained approaches aim at increasing the robustness of the estimates by adopting models in which an objective function is minimized through the definition of physiological constraints. This thesis proposed the methods to accurately estimate the human joint kinematics starting from the MIMU signals. Three main contributions were provided. The first consisted in the design of a comprehensive battery of tests to completely characterize the sources of errors affecting the quality of the measurements. These tests rely on simple hypotheses based on the sensor working principles and do not require expensive equipment. Nine parameters were defined to quantify the signal accuracy improvements (if any) of 24 MIMUs before and after the refinement of their calibration coefficients. The second contribution was focused on the SFAs. Ten among the most popular SFAs were compared under different experimental conditions including different MIMU models and rotation rate magnitudes. To perform a “fair” comparison it was necessary to set the optimal parameter values for each SFA. The most important finding was that all the errors fall within a range from 3.8 deg to 7.1 deg thus making it impossible to draw any conclusions about the best performing SFA since no statistically significant differences were found. In addition, the orientation accuracy was heavily influenced by the experimental variables. After that, a novel method was designed to estimate the suboptimal parameter values of a given SFA without relying on any orientation reference. The maximum difference between the errors obtained using optimal and suboptimal parameter values amounted to 3.7 deg and to 0.6 deg on average. The last contribution consisted in the design of an unconstrained and a constrained methods for estimating the joint kinematics without considering the magnetometer to avoid the ferromagnetic disturbances. The unconstrained method was employed in a telerehabilitation platform in which the joint angles were estimated in real time. Errors collected during the execution of a full-body protocol were lower than 5 deg (considered the acceptability threshold). However, this method may be inaccurate after few minutes since no solutions can be taken to mitigate the drift error. To overcome this limitation a constrained method was developed based on a robotic model of the upper limb to set appropriate constraints. Errors relative to a continuous robot motion for twenty minutes were lower than 3 deg at most suggesting the feasibility of employing these solutions in the rehabilitation programs to properly plan the treatment and to accurately evaluate the outcomes

Human Motion Analysis with Wearable Inertial Sensors

High-resolution, quantitative data obtained by a human motion capture system can be used to better understand the cause of many diseases for effective treatments. Talking about the daily care of the aging population, two issues are critical. One is to continuously track motions and position of aging people when they are at home, inside a building or in the unknown environment; the other is to monitor their health status in real time when they are in the free-living environment. Continuous monitoring of human movement in their natural living environment potentially provide more valuable feedback than these in laboratory settings. However, it has been extremely challenging to go beyond laboratory and obtain accurate measurements of human physical activity in free-living environments. Commercial motion capture systems produce excellent in-studio capture and reconstructions, but offer no comparable solution for acquisition in everyday environments. Therefore in this dissertation, a wearable human motion analysis system is developed for continuously tracking human motions, monitoring health status, positioning human location and recording the itinerary. In this dissertation, two systems are developed for seeking aforementioned two goals: tracking human body motions and positioning a human. Firstly, an inertial-based human body motion tracking system with our developed inertial measurement unit (IMU) is introduced. By arbitrarily attaching a wearable IMU to each segment, segment motions can be measured and translated into inertial data by IMUs. A human model can be reconstructed in real time based on the inertial data by applying high efficient twists and exponential maps techniques. Secondly, for validating the feasibility of developed tracking system in the practical application, model-based quantification approaches for resting tremor and lower extremity bradykinesia in Parkinson’s disease are proposed. By estimating all involved joint angles in PD symptoms based on reconstructed human model, angle characteristics with corresponding medical ratings are employed for training a HMM classifier for quantification. Besides, a pedestrian positioning system is developed for tracking user’s itinerary and positioning in the global frame. Corresponding tests have been carried out to assess the performance of each system

Auto-encoder based deep learning for surface electromyography signal processing

© 2018 Advances in Science, Technology and Engineering Systems. All Rights Reserved. Feature extraction is taking a very vital and essential part of bio-signal processing. We need to choose one of two paths to identify and select features in any system. The most popular track is engineering handcrafted, which mainly depends on the user experience and the field of application. While the other path is feature learning, which depends on training the system on recognising and picking the best features that match the application. The main concept of feature learning is to create a model that is expected to be able to learn the best features without any human intervention instead of recourse the traditional methods for feature extraction or reduction and avoid dealing with feature extraction that depends on researcher experience. In this paper, Auto-Encoder will be utilised as a feature learning algorithm to practice the recommended model to excerpt the useful features from the surface electromyography signal. Deep learning method will be suggested by using Auto-Encoder to learn features. Wavelet Packet, Spectrogram, and Wavelet will be employed to represent the surface electromyography signal in our recommended model. Then, the newly represented bio-signal will be fed to stacked autoencoder (2 stages) to learn features and finally, the behaviour of the proposed algorithm will be estimated by hiring different classifiers such as Extreme Learning Machine, Support Vector Machine, and SoftMax Layer. The Rectified Linear Unit (ReLU) will be created as an activation function for extreme learning machine classifier besides existing functions such as sigmoid and radial basis function. ReLU will show a better classification ability than sigmoid and Radial basis function (RBF) for wavelet, Wavelet scale 5 and wavelet packet signal representations implemented techniques. ReLU will illustrate better classification ability, as an activation function, than sigmoid and poorer than RBF for spectrogram signal representation. Both confidence interval and Analysis of Variance will be estimated for different classifiers. Classifier fusion layer will be implemented to glean the classifier which will progress the best accuracies' values for both testing and training to develop the results. Classifier fusion layer brought an encouraging value for both accuracies either training or testing ones

RGB-D And Thermal Sensor Fusion: A Systematic Literature Review

In the last decade, the computer vision field has seen significant progress in multimodal data fusion and learning, where multiple sensors, including depth, infrared, and visual, are used to capture the environment across diverse spectral ranges. Despite these advancements, there has been no systematic and comprehensive evaluation of fusing RGB-D and thermal modalities to date. While autonomous driving using LiDAR, radar, RGB, and other sensors has garnered substantial research interest, along with the fusion of RGB and depth modalities, the integration of thermal cameras and, specifically, the fusion of RGB-D and thermal data, has received comparatively less attention. This might be partly due to the limited number of publicly available datasets for such applications. This paper provides a comprehensive review of both, state-of-the-art and traditional methods used in fusing RGB-D and thermal camera data for various applications, such as site inspection, human tracking, fault detection, and others. The reviewed literature has been categorised into technical areas, such as 3D reconstruction, segmentation, object detection, available datasets, and other related topics. Following a brief introduction and an overview of the methodology, the study delves into calibration and registration techniques, then examines thermal visualisation and 3D reconstruction, before discussing the application of classic feature-based techniques as well as modern deep learning approaches. The paper concludes with a discourse on current limitations and potential future research directions. It is hoped that this survey will serve as a valuable reference for researchers looking to familiarise themselves with the latest advancements and contribute to the RGB-DT research field.Comment: 33 pages, 20 figure