Integration of Riemannian Motion Policy and Whole-Body Control for Dynamic Legged Locomotion

In this paper, we present a novel Riemannian Motion Policy (RMP)flow-based whole-body control framework for improved dynamic legged locomotion. RMPflow is a differential geometry-inspired algorithm for fusing multiple task-space policies (RMPs) into a configuration space policy in a geometrically consistent manner. RMP-based approaches are especially suited for designing simultaneous tracking and collision avoidance behaviors and have been successfully deployed on serial manipulators. However, one caveat of RMPflow is that it is designed with fully actuated systems in mind. In this work, we, for the first time, extend it to the domain of dynamic-legged systems, which have unforgiving under-actuation and limited control input. Thorough push recovery experiments are conducted in simulation to validate the overall framework. We show that expanding the valid stepping region with an RMP-based collision-avoidance swing leg controller improves balance robustness against external disturbances by up to $53\%$ compared to a baseline approach using a restricted stepping region. Furthermore, a point-foot biped robot is purpose-built for experimental studies of dynamic biped locomotion. A preliminary unassisted in-place stepping experiment is conducted to show the viability of the control framework and hardware

Beyond Basins of Attraction: Quantifying Robustness of Natural Dynamics

Properly designing a system to exhibit favorable natural dynamics can greatly simplify designing or learning the control policy. However, it is still unclear what constitutes favorable natural dynamics and how to quantify its effect. Most studies of simple walking and running models have focused on the basins of attraction of passive limit-cycles and the notion of self-stability. We instead emphasize the importance of stepping beyond basins of attraction. We show an approach based on viability theory to quantify robust sets in state-action space. These sets are valid for the family of all robust control policies, which allows us to quantify the robustness inherent to the natural dynamics before designing the control policy or specifying a control objective. We illustrate our formulation using spring-mass models, simple low dimensional models of running systems. We then show an example application by optimizing robustness of a simulated planar monoped, using a gradient-free optimization scheme. Both case studies result in a nonlinear effective stiffness providing more robustness.Comment: 15 pages. This work has been accepted to IEEE Transactions on Robotics (2019

Locomoção de humanoides robusta e versátil baseada em controlo analítico e física residual

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This thesis tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. We designed and developed model-based and model-free walk engines and formulated the controllers using different approaches including classical and optimal control schemes and validated their performance through simulations and experiments. These frameworks have hierarchical structures that are composed of several layers. These layers are composed of several modules that are connected together to fade the complexity and increase the flexibility of the proposed frameworks. Additionally, they can be easily and quickly deployed on different platforms. Besides, we believe that using machine learning on top of analytical approaches is a key to open doors for humanoid robots to step out of laboratories. We proposed a tight coupling between analytical control and deep reinforcement learning. We augmented our analytical controller with reinforcement learning modules to learn how to regulate the walk engine parameters (planners and controllers) adaptively and generate residuals to adjust the robot’s target joint positions (residual physics). The effectiveness of the proposed frameworks was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying human-like locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.Os robôs humanoides são feitos para se parecerem com humanos, mas suas habilidades de locomoção estão longe das nossas em termos de agilidade e versatilidade. Quando os humanos caminham em terrenos complexos ou enfrentam distúrbios externos combinam diferentes estratégias, de forma inconsciente e eficiente, para recuperar a estabilidade. Esta tese aborda o problema de desenvolver um sistema robusto para andar de forma omnidirecional, capaz de gerar uma locomoção para robôs humanoides versátil e ágil em terrenos complexos. Projetámos e desenvolvemos motores de locomoção sem modelos e baseados em modelos. Formulámos os controladores usando diferentes abordagens, incluindo esquemas de controlo clássicos e ideais, e validámos o seu desempenho por meio de simulações e experiências reais. Estes frameworks têm estruturas hierárquicas compostas por várias camadas. Essas camadas são compostas por vários módulos que são conectados entre si para diminuir a complexidade e aumentar a flexibilidade dos frameworks propostos. Adicionalmente, o sistema pode ser implementado em diferentes plataformas de forma fácil. Acreditamos que o uso de aprendizagem automática sobre abordagens analíticas é a chave para abrir as portas para robôs humanoides saírem dos laboratórios. Propusemos um forte acoplamento entre controlo analítico e aprendizagem profunda por reforço. Expandimos o nosso controlador analítico com módulos de aprendizagem por reforço para aprender como regular os parâmetros do motor de caminhada (planeadores e controladores) de forma adaptativa e gerar resíduos para ajustar as posições das juntas alvo do robô (física residual). A eficácia das estruturas propostas foi demonstrada e avaliada em um conjunto de cenários de simulação desafiadores. O robô foi capaz de generalizar o que aprendeu em um cenário, exibindo habilidades de locomoção humanas em circunstâncias imprevistas, mesmo na presença de ruído e impulsos externos.Programa Doutoral em Informátic

From Bipedal to Quadrupedal Locomotion, Experimental Realization of Lyapunov Approaches

Possibly one of the most significant innovations of the past decade is the hybrid zero dynamics (HZD) framework, which formally and rigorously designs a control algorithm for robotic walking. In this methodology, Lyapunov stability, which is often used to certificate a dynamical system's stability, was introduced to the control law design for a hybrid control system. However, the prerequisites of precise modeling to apply the HZD methodology can often be too restrictive to design controllers for uncertain and complex real-world hardware experiments. This thesis addresses the problem raised by noisy measurements and the intricate hybrid structure of locomotion dynamics. First, the HZD methodology's construction is based on the full-order, hybrid dynamics of legged locomotion, which can be intractable for control synthesis for high-dimensional systems. This thesis studies the general structure of hybrid control systems for walking systems, ranging from 1D hopping, 2D walking, 2D running, and 3D quadrupedal locomotion on rough terrains. Further, we characterize a walking behavior--gait--as a solution (execution) to a hybrid control system. To find these solutions, which represent a "gait," we employed advanced numerical methods such as collocation methods to parse the solution-finding problem into the open- and closed-loop trajectory optimization problems. The result is that we can find versatile gaits for ten different robotic platforms efficiently. This includes bipedal running, bipedal walking on slippery surfaces, and quadrupedal robots walking on sloped terrains. The numerous solution-finding examples expand the applicability of the HZD framework towards more complex dynamical systems. Further, for the uncertain and noisy real-world implementation, the exponential stability of the continuous dynamics is an ideal but restrictive condition for hybrid stability. This condition is especially challenging to satisfy for highly dynamical behaviors such as bipedal running, which loses ground support for a short period. This thesis observes the destabilizing effect of the noisy measurements of the phasing variable. By reformulating the traditional input-to-state stability (ISS) concept into phase-uncertainty to state stability, we are able to synthesize a robust controller for bipedal running on DURUS-2D. This time+state-based controller formally guarantees stability under noisy measurements and stabilizes the 1.75 m/s running experiments. Lastly, robotic dynamics have long been characterized as the interconnection of rigid-body dynamics. We take this perspective one step further and incorporate controller design into the formulation of coupled control systems (CCS). We first view a quadrupedal robot as two bipedal robots connected via some holonomic constraints. In a dimensional reduction manner, we develop a novel optimization framework, and the computational performance is reduced to a few seconds for gait generation. Furthermore, we can design local controllers for each bipedal subsystem and still guarantee the overall system's stability. This is done by combining the HZD framework and the ISS properties to contain the disturbance induced by the other subsystems' inputs. Utilizing the proposed CCS methods, we will experimentally realize quadrupedal walking on various outdoor rough terrains.</p

Climbing and Walking Robots

Nowadays robotics is one of the most dynamic fields of scientific researches. The shift of robotics researches from manufacturing to services applications is clear. During the last decades interest in studying climbing and walking robots has been increased. This increasing interest has been in many areas that most important ones of them are: mechanics, electronics, medical engineering, cybernetics, controls, and computers. Today’s climbing and walking robots are a combination of manipulative, perceptive, communicative, and cognitive abilities and they are capable of performing many tasks in industrial and non- industrial environments. Surveillance, planetary exploration, emergence rescue operations, reconnaissance, petrochemical applications, construction, entertainment, personal services, intervention in severe environments, transportation, medical and etc are some applications from a very diverse application fields of climbing and walking robots. By great progress in this area of robotics it is anticipated that next generation climbing and walking robots will enhance lives and will change the way the human works, thinks and makes decisions. This book presents the state of the art achievments, recent developments, applications and future challenges of climbing and walking robots. These are presented in 24 chapters by authors throughtot the world The book serves as a reference especially for the researchers who are interested in mobile robots. It also is useful for industrial engineers and graduate students in advanced study

Modular Hopping and Running via Parallel Composition

Though multi-functional robot hardware has been created, the complexity in its functionality has been constrained by a lack of algorithms that appropriately manage flexible and autonomous reconfiguration of interconnections to physical and behavioral components. Raibert pioneered a paradigm for the synthesis of planar hopping using a composition of ``parts\u27\u27: controlled vertical hopping, controlled forward speed, and controlled body attitude. Such reduced degree-of-freedom compositions also seem to appear in running animals across several orders of magnitude of scale. Dynamical systems theory can offer a formal representation of such reductions in terms of ``anchored templates,\u27\u27 respecting which Raibert\u27s empirical synthesis (and the animals\u27 empirical performance) can be posed as a parallel composition. However, the orthodox notion (attracting invariant submanifold with restriction dynamics conjugate to a template system) has only been formally synthesized in a few isolated instances in engineering (juggling, brachiating, hexapedal running robots, etc.) and formally observed in biology only in similarly limited contexts. In order to bring Raibert\u27s 1980\u27s work into the 21st century and out of the laboratory, we design a new family of one-, two-, and four-legged robots with high power density, transparency, and control bandwidth. On these platforms, we demonstrate a growing collection of $\{$body, behavior$\}$ pairs that successfully embody dynamical running / hopping ``gaits\u27\u27 specified using compositions of a few templates, with few parameters and a great deal of empirical robustness. We aim for and report substantial advances toward a formal notion of parallel composition---embodied behaviors that are correct by design even in the presence of nefarious coupling and perturbation---using a new analytical tool (hybrid dynamical averaging). With ideas of verifiable behavioral modularity and a firm understanding of the hardware tools required to implement them, we are closer to identifying the components required to flexibly program the exchange of work between machines and their environment. Knowing how to combine and sequence stable basins to solve arbitrarily complex tasks will result in improved foundations for robotics as it goes from ad-hoc practice to science (with predictive theories) in the next few decades

Viability in State-Action Space: Connecting Morphology, Control, and Learning

Wie können wir Robotern ermöglichen, modellfrei und direkt auf der Hardware zu lernen? Das maschinelle Lernen nimmt als Standardwerkzeug im Arsenal des Robotikers seinen Platz ein. Es gibt jedoch einige offene Fragen, wie man die Kontrolle über physikalische Systeme lernen kann. Diese Arbeit gibt zwei Antworten auf diese motivierende Frage. Das erste ist ein formales Mittel, um die inhärente Robustheit eines gegebenen Systemdesigns zu quantifizieren, bevor der Controller oder das Lernverfahren entworfen wird. Dies unterstreicht die Notwendigkeit, sowohl das Hardals auch das Software-Design eines Roboters zu berücksichtigen, da beide Aspekte in der Systemdynamik untrennbar miteinander verbunden sind. Die zweite ist die Formalisierung einer Sicherheitsmass, die modellfrei erlernt werden kann. Intuitiv zeigt diese Mass an, wie leicht ein Roboter Fehlschläge vermeiden kann. Auf diese Weise können Roboter unbekannte Umgebungen erkunden und gleichzeitig Ausfälle vermeiden. Die wichtigsten Beiträge dieser Dissertation basieren sich auf der Viabilitätstheorie. Viabilität bietet eine alternative Sichtweise auf dynamische Systeme: Anstatt sich auf die Konvergenzeigenschaften eines Systems in Richtung Gleichgewichte zu konzentrieren, wird der Fokus auf Menge von Fehlerzuständen und die Fähigkeit des Systems, diese zu vermeiden, verlagert. Diese Sichtweise eignet sich besonders gut für das Studium der Lernkontrolle an Robotern, da Stabilität im Sinne einer Konvergenz während des Lernprozesses selten gewährleistet werden kann. Der Begriff der Viabilität wird formal auf den Zustand-Aktion-Raum erweitert, mit Viabilitätsmengen von Staat-Aktionspaaren. Eine über diese Mengen definierte Mass ermöglicht eine quantifizierte Bewertung der Robustheit, die für die Familie aller fehlervermeidenden Regler gilt, und ebnet den Weg für ein sicheres, modellfreies Lernen. Die Arbeit beinhaltet auch zwei kleinere Beiträge. Der erste kleine Beitrag ist eine empirische Demonstration der Shaping durch ausschliessliche Modifikation der Systemdynamik. Diese Demonstration verdeutlicht die Bedeutung der Robustheit gegenüber Fehlern für die Lernkontrolle: Ausfälle können nicht nur Schäden verursachen, sondern liefern in der Regel auch keine nützlichen Gradienteninformationen für den Lernprozess. Der zweite kleine Beitrag ist eine Studie über die Wahl der Zustandsinitialisierungen. Entgegen der Intuition und der üblichen Praxis zeigt diese Studie, dass es zuverlässiger sein kann, das System gelegentlich aus einem Zustand zu initialisieren, der bekanntermassen unkontrollierbar ist.How can we enable robots to learn control model-free and directly on hardware? Machine learning is taking its place as a standard tool in the roboticist’s arsenal. However, there are several open questions on how to learn control for physical systems. This thesis provides two answers to this motivating question. The first is a formal means to quantify the inherent robustness of a given system design, prior to designing the controller or learning agent. This emphasizes the need to consider both the hardware and software design of a robot, which are inseparably intertwined in the system dynamics. The second is the formalization of a safety-measure, which can be learned model-free. Intuitively, this measure indicates how easily a robot can avoid failure, and enables robots to explore unknown environments while avoiding failures. The main contributions of this dissertation are based on viability theory. Viability theory provides a slightly unconventional view of dynamical systems: instead of focusing on a system’s convergence properties towards equilibria, the focus is shifted towards sets of failure states and the system’s ability to avoid these sets. This view is particularly well suited to studying learning control in robots, since stability in the sense of convergence can rarely be guaranteed during the learning process. The notion of viability is formally extended to state-action space, with viable sets of state-action pairs. A measure defined over these sets allows a quantified evaluation of robustness valid for the family of all failure-avoiding control policies, and also paves the way for enabling safe model-free learning. The thesis also includes two minor contributions. The first minor contribution is an empirical demonstration of shaping by exclusively modifying the system dynamics. This demonstration highlights the importance of robustness to failures for learning control: not only can failures cause damage, but they typically do not provide useful gradient information for the learning process. The second minor contribution is a study on the choice of state initializations. Counter to intuition and common practice, this study shows it can be more reliable to occasionally initialize the system from a state that is known to be uncontrollable