Online Trajectory Planning Through Combined Trajectory Optimization and Function Approximation: Application to the Exoskeleton Atalante

Autonomous robots require online trajectory planning capability to operate in the real world. Efficient offline trajectory planning methods already exist, but are computationally demanding, preventing their use online. In this paper, we present a novel algorithm called Guided Trajectory Learning that learns a function approximation of solutions computed through trajectory optimization while ensuring accurate and reliable predictions. This function approximation is then used online to generate trajectories. This algorithm is designed to be easy to implement, and practical since it does not require massive computing power. It is readily applicable to any robotics systems and effortless to set up on real hardware since robust control strategies are usually already available. We demonstrate the computational performance of our algorithm on flat-foot walking with the self-balanced exoskeleton Atalante

Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

State-of-the-art reinforcement learning is now able to learn versatile locomotion, balancing and push-recovery capabilities for bipedal robots in simulation. Yet, the reality gap has mostly been overlooked and the simulated results hardly transfer to real hardware. Either it is unsuccessful in practice because the physics is over-simplified and hardware limitations are ignored, or regularity is not guaranteed, and unexpected hazardous motions can occur. This paper presents a reinforcement learning framework capable of learning robust standing push recovery for bipedal robots that smoothly transfer to reality, providing only instantaneous proprioceptive observations. By combining original termination conditions and policy smoothness conditioning, we achieve stable learning, sim-to-real transfer and safety using a policy without memory nor explicit history. Reward engineering is then used to give insights into how to keep balance. We demonstrate its performance in reality on the lower-limb medical exoskeleton Atalante

Towards Variable Assistance for Lower Body Exoskeletons

This letter presents and experimentally demonstrates a novel framework for variable assistance on lower body exoskeletons, based upon safety-critical control methods. Existing work has shown that providing some freedom of movement around a nominal gait, instead of rigidly following it, accelerates the spinal learning process of people with a walking impediment when using a lower body exoskeleton. With this as motivation, we present a method to accurately control how much a subject is allowed to deviate from a given gait while ensuring robustness to patient perturbation. This method leverages control barrier functions to force certain joints to remain inside predefined trajectory tubes in a minimally invasive way. The effectiveness of the method is demonstrated experimentally with able-bodied subjects and the Atalante lower body exoskeleton

Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable Hands-free Dynamic Walking

This manuscript presents control of a high-DOF fully actuated lower-limb exoskeleton for paraplegic individuals. The key novelty is the ability for the user to walk without the use of crutches or other external means of stabilization. We harness the power of modern optimization techniques and supervised machine learning to develop a smooth feedback control policy that provides robust velocity regulation and perturbation rejection. Preliminary evaluation of the stability and robustness of the proposed approach is demonstrated through the Gazebo simulation environment. In addition, preliminary experimental results with (complete) paraplegic individuals are included for the previous version of the controller.Comment: Submitted to IEEE Control System Magazine. This version addresses reviewers' concerns about the robustness of the algorithm and the motivation for using such exoskeleton

Interactive locomotion: Investigation and modeling of physically-paired humans while walking

In spite of extensive studies on human walking, less research has been conducted on human walking gait adaptation during interaction with another human. In this paper, we study a particular case of interactive locomotion where two humans carry a rigid object together. Experimental data from two persons walking together, one in front of the other, while carrying a stretcher-like object is presented, and the adaptation of their walking gaits and coordination of the foot-fall patterns are analyzed. It is observed that in more than 70% of the experiments the subjects synchronize their walking gaits; it is shown that these walking gaits can be associated to quadrupedal gaits. Moreover, in order to understand the extent by which the passive dynamics can explain this synchronization behaviour, a simple 2D model, made of two-coupled spring-loaded inverted pendulums, is developed, and a comparison between the experiments and simulations with this model is presented, showing that with this simple model we are able to reproduce some aspects of human walking behaviour when paired with another human

Feedback Control of an Exoskeleton for Paraplegics: Toward Robustly Stable, Hands-Free Dynamic Walking

"I will never forget the emotion of my first steps […]," were the words of Fran?oise, the first user during initial trials of the exoskeleton ATALANTE [1]. "I am tall again!" were the words of Sandy (the fourth user) after standing up in the exoskeleton. During these early tests, complete paraplegic patients dynamically walked up to 10 m without crutches or other assistance using a feedback control method originally invented for bipedal robots. As discussed in "Summary," this article describes the hardware (shown in Figure 1) that was designed to achieve hands-free dynamic walking, the control laws that were deployed (and those being developed) to provide enhanced mobility and robustness, and preliminary test results. In this article, dynamic walking refers to a motion that is orbitally stable as opposed to statically stable

Learning and Optimization of the Locomotion with an Exoskeleton for Paraplegic People

Cette thèse contribue à améliorer la planification de trajectoires et le contrôle des robots bipèdes. Le but concret est de permettre aux paraplégiques de remarcher de façon autonome avec l’exosquelette de membres inférieurs Atalante. Notre approche combine les méthodes issues l’apprentissage automatique et de la robotique traditionnelle. Nous mettons d’abord de côté le contrôle. L’objectif est de permettre la planification de trajectoires en ligne tout en garantissant un fonctionnement sûr. C’est une étape cruciale vers la navigation en milieu incertain et la prise en compte des préférences utilisateur. Nous entraînons ensuite un contrôleur par renforcement afin de généraliser un ensemble prédéfini de mouvements élémentaires. Nous ne cherchons pas la meilleure performance, mais plutôt la transférabilité et la sécurité. Nous proposons une formulation qui apparente à l’apprentissage par imitation mais laisse suffisamment de marge de manœuvre pour affronter des événements inattendus.This thesis contributes to improving the motion planning and control of biped robots. Our concrete goal is restoring natural locomotion for paraplegic people in their daily lives using the medical lower-limb exoskeleton Atalante, notably walkingsafely and autonomously without crutches. The core idea is to combine traditional robotics and state-of-the-art machine learning. We put aside closed-loop control to focus on planning at first. The objective is to enable online trajectory planning while ensuring safe operation. This is a milestone toward realizing versatile navigation in an unstructured environment and accommodating the user preferences. Second, we train a policy using reinforcement learning to generalize a predefined set of primitive motions. We do not seek the best possible performance, but rather transferability and safety. We propose a formulation closely related to imitation learning while giving enough leeway to deal with unexpected events

Apprentissage et Optimisation de la Locomotion pour un Exosquelette à destination des Patients Paraplégiques

This thesis contributes to improving the motion planning and control of biped robots. Our concrete goal is restoring natural locomotion for paraplegic people in their daily lives using the medical lower-limb exoskeleton Atalante, notably walkingsafely and autonomously without crutches. The core idea is to combine traditional robotics and state-of-the-art machine learning. We put aside closed-loop control to focus on planning at first. The objective is to enable online trajectory planning while ensuring safe operation. This is a milestone toward realizing versatile navigation in an unstructured environment and accommodating the user preferences. Second, we train a policy using reinforcement learning to generalize a predefined set of primitive motions. We do not seek the best possible performance, but rather transferability and safety. We propose a formulation closely related to imitation learning while giving enough leeway to deal with unexpected events.Cette thèse contribue à améliorer la planification de trajectoires et le contrôle des robots bipèdes. Le but concret est de permettre aux paraplégiques de remarcher de façon autonome avec l’exosquelette de membres inférieurs Atalante. Notre approche combine les méthodes issues l’apprentissage automatique et de la robotique traditionnelle. Nous mettons d’abord de côté le contrôle. L’objectif est de permettre la planification de trajectoires en ligne tout en garantissant un fonctionnement sûr. C’est une étape cruciale vers la navigation en milieu incertain et la prise en compte des préférences utilisateur. Nous entraînons ensuite un contrôleur par renforcement afin de généraliser un ensemble prédéfini de mouvements élémentaires. Nous ne cherchons pas la meilleure performance, mais plutôt la transférabilité et la sécurité. Nous proposons une formulation qui apparente à l’apprentissage par imitation mais laisse suffisamment de marge de manœuvre pour affronter des événements inattendus

Interactive Locomotion of Mechanically Coupled Bipedal Agents: Modeling and Experiments

This paper investigates the interactive locomotion for physically coupled bipedal agents using a human-human object carrying experiment and a simple mathematical model. The model is based on the Spring Loaded Inverted Pendulum (SLIP) and on the assumption that the coupling can be modeled as a spring-damper. By setting appropriate parameters, the model can achieve stable walking gaits and coordinated foot-fall patterns. Human-human interactive locomotion data are also analyzed in order to evaluate how the model can be useful to investigate humans' interaction. The kinematic study of how the rigid coupling between two humans can influence their own behaviour is presented for the first time to our knowledge. Moreover a good match between experimental and model data emerges from the comparison between several gait parameters.BIORO