A Bio-inspired architecture for adaptive quadruped locomotion over irregular terrain

Tese de doutoramento Programa Doutoral em Engenharia Electrónica e de ComputadoresThis thesis presents a tentative advancement on walking control of small quadruped and humanoid position controlled robots, addressing the problem of walk generation by combining dynamical systems approach to motor control, insights from neuroethology research on vertebrate motor control and computational neuroscience. Legged locomotion is a complex dynamical process, despite the seemingly easy and natural behavior of the constantly present proficiency of legged animals. Research on locomotion and motor control in vertebrate animals from the last decades has brought to the attention of roboticists, the potential of the nature’s solutions to robot applications. Recent knowledge on the organization of complex motor generation and on mechanics and dynamics of locomotion has been successfully exploited to pursue agile robot locomotion. The work presented on this manuscript is part of an effort on the pursuit in devising a general, model free solution, for the generation of robust and adaptable walking behaviors. It strives to devise a practical solution applicable to real robots, such as the Sony’s quadruped AIBO and Robotis’ DARwIn- OP humanoid. The discussed solutions are inspired on the functional description of the vertebrate neural systems, especially on the concept of Central Pattern Generators (CPGs), their structure and organization, components and sensorimotor interactions. They use a dynamical systems approach for the implementation of the controller, especially on the use of nonlinear oscillators and exploitation of their properties. The main topics of this thesis are divided into three parts. The first part concerns quadruped locomotion, extending a previous CPG solution using nonlinear oscillators, and discussing an organization on three hierarchical levels of abstraction, sharing the purpose and knowledge of other works. It proposes a CPG solution which generates the walking motion for the whole-leg, which is then organized in a network for the production of quadrupedal gaits. The devised solution is able to produce goal-oriented locomotion and navigation as directed through highlevel commands from local planning methods. In this part, active balance on a standing quadruped is also addressed, proposing a method based on dynamical systems approach, exploring the integration of parallel postural mechanisms from several sensory modalities. The solutions are all successfully tested on the quadruped AIBO robot. In the second part, is addressed bipedal walking for humanoid robots. A CPG solution for biped walking based on the concept of motion primitives is proposed, loosely based on the idea of synergistic organization of vertebrate motor control. A set of motion primitives is shown to produce the basis of simple biped walking, and generalizable to goal-oriented walking. Using the proposed CPG, the inclusion of feedback mechanisms is investigated, for modulation and adaptation of walking, through phase transition control according to foot load information. The proposed solution is validated on the humanoid DARwIn-OP, and its application is evaluated within a whole-body control framework. The third part sidesteps a little from the other two topics. It discusses the CPG as having an alternative role to direct motor generation in locomotion, serving instead as a processor of sensory information for a feedback based motor generation. In this work a reflex based walking controller is devised for the compliant quadruped Oncilla robot, to serve as purely feedback based walking generation. The capabilities of the reflex network are shown in simulations, followed by a brief discussion on its limitations, and how they could be improved by the inclusion of a CPG.Esta tese apresenta uma tentativa de avanço no controlo de locomoção para pequenos robôs quadrúpedes e bipedes controlados por posição, endereçando o problema de geração motora através da combinação da abordagem de sistemas dinâmicos para o controlo motor, e perspectivas de investigação neuroetologia no controlo motor vertebrado e neurociência computacional. Andar é um processo dinâmico e complexo, apesar de parecer um comportamento fácil e natural devido à presença constante de animais proficientes em locomoção terrestre. Investigação na área da locomoção e controlo motor em animais vertebrados nas últimas decadas, trouxe à atenção dos roboticistas o potencial das soluções encontradas pela natureza aplicadas a aplicações robóticas. Conhecimento recente relativo à geração de comportamentos motores complexos e da mecânica da locomoção tem sido explorada com sucesso na procura de locomoção ágil na robótica. O trabalho apresentado neste documento é parte de um esforço no desenho de uma solução geral, e independente de modelos, para a geração robusta e adaptável de comportamentos locomotores. O foco é desenhar uma solução prática, aplicável a robôs reais, tal como o quadrúpede Sony AIBO e o humanóide DARwIn-OP. As soluções discutidas são inspiradas na descrição funcional do sistema nervoso vertebrado, especialmente no conceito de Central Pattern Generators (CPGs), a sua estrutura e organização, componentes e interacção sensorimotora. Estas soluções são implementadas usando uma abordagem em sistemas dinâmicos, focandos o uso de osciladores não lineares e a explorando as suas propriedades. Os tópicos principais desta tese estão divididos em três partes. A primeira parte explora o tema de locomoção quadrúpede, expandindo soluções prévias de CPGs usando osciladores não lineares, e discutindo uma organização em três níveis de abstracção, partilhando as ideias de outros trabalhos. Propõe uma solução de CPG que gera os movimentos locomotores para uma perna, que é depois organizado numa rede, para a produção de marcha quadrúpede. A solução concebida é capaz de produzir locomoção e navegação, comandada através de comandos de alto nível, produzidos por métodos de planeamento local. Nesta parte também endereçado o problema da manutenção do equilíbrio num robô quadrúpede parado, propondo um método baseado na abordagem em sistemas dinâmicos, explorando a integração de mecanismos posturais em paralelo, provenientes de várias modalidades sensoriais. As soluções são todas testadas com sucesso no robô quadrupede AIBO. Na segunda parte é endereçado o problema de locomoção bípede. É proposto um CPG baseado no conceito de motion primitives, baseadas na ideia de uma organização sinergética do controlo motor vertebrado. Um conjunto de motion primitives é usado para produzir a base de uma locomoção bípede simples e generalizável para navegação. Esta proposta de CPG é usada para de seguida se investigar a inclusão de mecanismos de feedback para modulação e adaptação da marcha, através do controlo de transições entre fases, de acordo com a informação de carga dos pés. A solução proposta é validada no robô humanóide DARwIn-OP, e a sua aplicação no contexto do framework de whole-body control é também avaliada. A terceira parte desvia um pouco dos outros dois tópicos. Discute o CPG como tendo um papel alternativo ao controlo motor directo, servindo em vez como um processador de informação sensorial para um mecanismo de locomoção puramente em feedback. Neste trabalho é desenhado um controlador baseado em reflexos para a geração da marcha de um quadrúpede compliant. As suas capacidades são demonstradas em simulação, seguidas por uma breve discussão nas suas limitações, e como estas podem ser ultrapassadas pela inclusão de um CPG.The presented work was possible thanks to the support by the Portuguese Science and Technology Foundation through the PhD grant SFRH/BD/62047/2009

Fast biped walking with a neuronal controller and physical computation

Biped walking remains a difficult problem and robot models can greatly {facilitate} our understanding of the underlying biomechanical principles as well as their neuronal control. The goal of this study is to specifically demonstrate that stable biped walking can be achieved by combining the physical properties of the walking robot with a small, reflex-based neuronal network, which is governed mainly by local sensor signals. This study shows that human-like gaits emerge without {specific} position or trajectory control and that the walker is able to compensate small disturbances through its own dynamical properties. The reflexive controller used here has the following characteristics, which are different from earlier approaches: (1) Control is mainly local. Hence, it uses only two signals (AEA=Anterior Extreme Angle and GC=Ground Contact) which operate at the inter-joint level. All other signals operate only at single joints. (2) Neither position control nor trajectory tracking control is used. Instead, the approximate nature of the local reflexes on each joint allows the robot mechanics itself (e.g., its passive dynamics) to contribute substantially to the overall gait trajectory computation. (3) The motor control scheme used in the local reflexes of our robot is more straightforward and has more biological plausibility than that of other robots, because the outputs of the motorneurons in our reflexive controller are directly driving the motors of the joints, rather than working as references for position or velocity control. As a consequence, the neural controller and the robot mechanics are closely coupled as a neuro-mechanical system and this study emphasises that dynamically stable biped walking gaits emerge from the coupling between neural computation and physical computation. This is demonstrated by different walking experiments using two real robot as well as by a Poincar\'{e} map analysis applied on a model of the robot in order to assess its stability. In addition, this neuronal control structure allows the use of a policy gradient reinforcement learning algorithm to tune the parameters of the neurons in real-time, during walking. This way the robot can reach a record-breaking walking speed of 3.5 leg-lengths per second after only a few minutes of online learning, which is even comparable to the fastest relative speed of human walking

Adapting Highly-Dynamic Compliant Movements to Changing Environments: A Benchmark Comparison of Reflex- vs. CPG-Based Control Strategies

To control highly-dynamic compliant motions such as running or hopping, vertebrates rely on reflexes and Central Pattern Generators (CPGs) as core strategies. However, decoding how much each strategy contributes to the control and how they are adjusted under different conditions is still a major challenge. To help solve this question, the present paper provides a comprehensive comparison of reflexes, CPGs and a commonly used combination of the two applied to a biomimetic robot. It leverages recent findings indicating that in mammals both control principles act within a low-dimensional control submanifold. This substantially reduces the search space of parameters and enables the quantifiable comparison of the different control strategies. The chosen metrics are motion stability and energy efficiency, both key aspects for the evolution of the central nervous system. We find that neither for stability nor energy efficiency it is favorable to apply the state-of-the-art approach of a continuously feedback-adapted CPG. In both aspects, a pure reflex is more effective, but the pure CPG allows easy signal alteration when needed. Additionally, the hardware experiments clearly show that the shape of a control signal has a strong influence on energy efficiency, while previous research usually only focused on frequency alignment. Both findings suggest that currently used methods to combine the advantages of reflexes and CPGs can be improved. In future research, possible combinations of the control strategies should be reconsidered, specifically including the modulation of the control signal's shape. For this endeavor, the presented setup provides a valuable benchmark framework to enable the quantitative comparison of different bioinspired control principles

Locomoção bípede adaptativa a partir de uma única demonstração usando primitivas de movimento

Doutoramento em Engenharia EletrotécnicaEste trabalho aborda o problema de capacidade de imitação da locomoção humana através da utilização de trajetórias de baixo nível codificadas com primitivas de movimento e utilizá-las para depois generalizar para novas situações, partindo apenas de uma demonstração única. Assim, nesta linha de pensamento, os principais objetivos deste trabalho são dois: o primeiro é analisar, extrair e codificar demonstrações efetuadas por um humano, obtidas por um sistema de captura de movimento de forma a modelar tarefas de locomoção bípede. Contudo, esta transferência não está limitada à simples reprodução desses movimentos, requerendo uma evolução das capacidades para adaptação a novas situações, assim como lidar com perturbações inesperadas. Assim, o segundo objetivo é o desenvolvimento e avaliação de uma estrutura de controlo com capacidade de modelação das ações, de tal forma que a demonstração única apreendida possa ser modificada para o robô se adaptar a diversas situações, tendo em conta a sua dinâmica e o ambiente onde está inserido. A ideia por detrás desta abordagem é resolver o problema da generalização a partir de uma demonstração única, combinando para isso duas estruturas básicas. A primeira consiste num sistema gerador de padrões baseado em primitivas de movimento utilizando sistemas dinâmicos (DS). Esta abordagem de codificação de movimentos possui propriedades desejáveis que a torna ideal para geração de trajetórias, tais como a possibilidade de modificar determinados parâmetros em tempo real, tais como a amplitude ou a frequência do ciclo do movimento e robustez a pequenas perturbações. A segunda estrutura, que está embebida na anterior, é composta por um conjunto de osciladores acoplados em fase que organizam as ações de unidades funcionais de forma coordenada. Mudanças em determinadas condições, como o instante de contacto ou impactos com o solo, levam a modelos com múltiplas fases. Assim, em vez de forçar o movimento do robô a situações pré-determinadas de forma temporal, o gerador de padrões de movimento proposto explora a transição entre diferentes fases que surgem da interação do robô com o ambiente, despoletadas por eventos sensoriais. A abordagem proposta é testada numa estrutura de simulação dinâmica, sendo que várias experiências são efetuadas para avaliar os métodos e o desempenho dos mesmos.This work addresses the problem of learning to imitate human locomotion actions through low-level trajectories encoded with motion primitives and generalizing them to new situations from a single demonstration. In this line of thought, the main objectives of this work are twofold: The first is to analyze, extract and encode human demonstrations taken from motion capture data in order to model biped locomotion tasks. However, transferring motion skills from humans to robots is not limited to the simple reproduction, but requires the evaluation of their ability to adapt to new situations, as well as to deal with unexpected disturbances. Therefore, the second objective is to develop and evaluate a control framework for action shaping such that the single-demonstration can be modulated to varying situations, taking into account the dynamics of the robot and its environment. The idea behind the approach is to address the problem of generalization from a single-demonstration by combining two basic structures. The first structure is a pattern generator system consisting of movement primitives learned and modelled by dynamical systems (DS). This encoding approach possesses desirable properties that make them well-suited for trajectory generation, namely the possibility to change parameters online such as the amplitude and the frequency of the limit cycle and the intrinsic robustness against small perturbations. The second structure, which is embedded in the previous one, consists of coupled phase oscillators that organize actions into functional coordinated units. The changing contact conditions plus the associated impacts with the ground lead to models with multiple phases. Instead of forcing the robot’s motion into a predefined fixed timing, the proposed pattern generator explores transition between phases that emerge from the interaction of the robot system with the environment, triggered by sensor-driven events. The proposed approach is tested in a dynamics simulation framework and several experiments are conducted to validate the methods and to assess the performance of a humanoid robot

Robot-driven epidural spinal cord stimulation compared with conventional stimulation in adult spinalized rats

Epidural stimulation to trigger locomotion is a promising treatment after spinal cord injury (SCI). Continuous stimulation during locomotion is the conventional method. To improve recovery, we designed and tested an innovative robot-driven epidural stimulation method, coupled with a trunk-based neurorobotic system. The system was tested in rats, and the results were compared with the results of the neurorobotic therapy combined with the conventional epidural stimulation method, and with robotic rehabilitation alone. The rats had better recovery after treatment with the robot-driven epidural stimulation than conventional stimulation or controls in our neurorobotic rehabilitation system.Ph.D., Biomedical Engineering -- Drexel University, 201

Combining Reflexes and External Sensory Information in a Neuromusculoskeletal Model to Control a Quadruped Robot

This article examines the importance of integrating locomotion and cognitive information for achieving dynamic locomotion from a viewpoint combining biology and ecological psychology. We present a mammalian neuromusculoskeletal model from external sensory information processing to muscle activation, which includes: 1) a visual-attention control mechanism for controlling attention to external inputs; 2) object recognition representing the primary motor cortex; 3) a motor control model that determines motor commands traveling down the corticospinal and reticulospinal tracts; 4) a central pattern generation model representing pattern generation in the spinal cord; and 5) a muscle reflex model representing the muscle model and its reflex mechanism. The proposed model is able to generate the locomotion of a quadruped robot in flat and natural terrain. The experiment also shows the importance of a postural reflex mechanism when experiencing a sudden obstacle. We show the reflex mechanism when a sudden obstacle is separately detected from both external (retina) and internal (touching afferent) sensory information. We present the biological rationale for supporting the proposed model. Finally, we discuss future contributions, trends, and the importance of the proposed research

Multilevel Analysis of Locomotion in Immature Preparations Suggests Innovative Strategies to Reactivate Stepping after Spinal Cord Injury

Locomotion is one of the most complex motor behaviors. Locomotor patterns change during early life, reflecting development of numerous peripheral and hierarchically organized central structures. Among them, the spinal cord is of particular interest since it houses the central pattern generator (CPG) for locomotion. This main command center is capable of eliciting and coordinating complex series of rhythmic neural signals sent to motoneurons and to corresponding target-muscles for basic locomotor activity. For a long-time, the CPG has been considered a black box. In recent years, complementary insights from in vitro and in vivo animal models have contributed significantly to a better understanding of its constituents, properties and ways to recover locomotion after a spinal cord injury (SCI). This review discusses key findings made by comparing the results of in vitro isolated spinal cord preparations and spinal-transected in vivo models from neonatal animals. Pharmacological, electrical, and sensory stimulation approaches largely used to further understand CPG function may also soon become therapeutic tools for potent CPG reactivation and locomotor movement induction in persons with SCI or developmental neuromuscular disorder

Neuroprosthetic system to restore locomotion after neuromotor disorder

Neuromodulation of spinal sensorimotor circuits improves motor control in animal models and humans with Spinal Cord Injury (SCI) and Parkinson disease. Stimulation parameters are tuned manually and remain constant during motor execution which is suboptimal to mediate maximum therapeutic effects. Here, I present a novel neuroprosthetic system that enabled adaptive changes of neuromodulation parameters during locomotion and allowed to restore high-fidelity control over leg movements in paralyzed rats. Beyond the therapeutic potential, these findings provide a conceptual and technical framework to personalize neuromodulation treatments for other neurological disorders. Several limitations have restricted the development of neuroprosthetic systems for closed loop neuromodulation. (1) First, it required a mechanistic understanding of the relationships between stimulation features and the recruitment of specific sensorimotor circuits. I found that electrical neuromodulation primarily recruits afferent reflex pathways that lead to coordinated activity of leg muscles during stepping. Moreover, the specific electrode location on the spinal cord could activate distinct reflex pathways and activate specific leg muscle groups of paralyzed rats. These results have been leveraged for the design of flexible and stretchable multi-electrode arrays for electrical and chemical spinal cord stimulation. (2) Second, it was necessary to perform comprehensive mapping experiments to characterize the effect of neuromodulation parameters on hind limb kinematics in order to establish stable and robust feedback signals for real time control. Step height and ground reaction forces emerged as the primary targets for the control of closed loop neuromodulation after spinal cord injury. (3) Third, implementation and optimization of closed-loop neuromodulation strategies necessitated the development of an advanced technological platform that combined feedback and feed-forward loops that match the natural flow of information in the modulated neural systems. These integrated developments allowed animals with complete spinal cord injury to perform over 1000 successive steps without failure, and to climb staircases of various heights and lengths with precision and fluidity. Moreover, the neuroprosthetic system was able to alleviate locomotor deficits in an alpha-synuclein rodent model of Parkinsonâs disease. Current knowledge of human spinal cord properties in response to electrical neuromodulation suggests that the developed control policies can translate into clinical applications to improve neurorehabilitation therapies. Moreover, the developed neuroprosthetic system can readily be interfaced with control signals from the brain to establish cortico-spinal neuroprostheses that are intended to promote activity-dependent plasticity during recovery from spinal cord injury

Learning control of bipedal dynamic walking robots with neural networks

Thesis (Elec.E.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998.Includes bibliographical references (p. 90-94).Stability and robustness are two important performance requirements for a dynamic walking robot. Learning and adaptation can improve stability and robustness. This thesis explores such an adaptation capability through the use of neural networks. Three neural network models (BP, CMAC and RBF networks) are studied. The RBF network is chosen as best, despite its weakness at covering high dimensional input spaces. To overcome this problem, a self-organizing scheme of data clustering is explored. This system is applied successfully in a biped walking robot system with a supervised learning mode. Generalized Virtual Model Control (GVMC) is also proposed in this thesis, which is inspired by a bio-mechanical model of locomotion, and is an extension of ordinary Virtual Model Control. Instead of adding virtual impedance components to the biped skeletal system in virtual Cartesian space, GVMC uses adaptation to approximately reconstruct the dynamics of the biped. The effectiveness of these approaches is proved both theoretically and experimentally (in simulation).by Jianjuen Hu.Elec.E