Evolution of central pattern generators for the control of a five-link bipedal walking mechanism

Central pattern generators (CPGs), with a basis is neurophysiological studies, are a type of neural network for the generation of rhythmic motion. While CPGs are being increasingly used in robot control, most applications are hand-tuned for a specific task and it is acknowledged in the field that generic methods and design principles for creating individual networks for a given task are lacking. This study presents an approach where the connectivity and oscillatory parameters of a CPG network are determined by an evolutionary algorithm with fitness evaluations in a realistic simulation with accurate physics. We apply this technique to a five-link planar walking mechanism to demonstrate its feasibility and performance. In addition, to see whether results from simulation can be acceptably transferred to real robot hardware, the best evolved CPG network is also tested on a real mechanism. Our results also confirm that the biologically inspired CPG model is well suited for legged locomotion, since a diverse manifestation of networks have been observed to succeed in fitness simulations during evolution.Comment: 11 pages, 9 figures; substantial revision of content, organization, and quantitative result

Locomoção bípede adaptativa a partir de uma única demonstração usando primitivas de movimento

Doutoramento em Engenharia EletrotécnicaEste trabalho aborda o problema de capacidade de imitação da locomoção humana através da utilização de trajetórias de baixo nível codificadas com primitivas de movimento e utilizá-las para depois generalizar para novas situações, partindo apenas de uma demonstração única. Assim, nesta linha de pensamento, os principais objetivos deste trabalho são dois: o primeiro é analisar, extrair e codificar demonstrações efetuadas por um humano, obtidas por um sistema de captura de movimento de forma a modelar tarefas de locomoção bípede. Contudo, esta transferência não está limitada à simples reprodução desses movimentos, requerendo uma evolução das capacidades para adaptação a novas situações, assim como lidar com perturbações inesperadas. Assim, o segundo objetivo é o desenvolvimento e avaliação de uma estrutura de controlo com capacidade de modelação das ações, de tal forma que a demonstração única apreendida possa ser modificada para o robô se adaptar a diversas situações, tendo em conta a sua dinâmica e o ambiente onde está inserido. A ideia por detrás desta abordagem é resolver o problema da generalização a partir de uma demonstração única, combinando para isso duas estruturas básicas. A primeira consiste num sistema gerador de padrões baseado em primitivas de movimento utilizando sistemas dinâmicos (DS). Esta abordagem de codificação de movimentos possui propriedades desejáveis que a torna ideal para geração de trajetórias, tais como a possibilidade de modificar determinados parâmetros em tempo real, tais como a amplitude ou a frequência do ciclo do movimento e robustez a pequenas perturbações. A segunda estrutura, que está embebida na anterior, é composta por um conjunto de osciladores acoplados em fase que organizam as ações de unidades funcionais de forma coordenada. Mudanças em determinadas condições, como o instante de contacto ou impactos com o solo, levam a modelos com múltiplas fases. Assim, em vez de forçar o movimento do robô a situações pré-determinadas de forma temporal, o gerador de padrões de movimento proposto explora a transição entre diferentes fases que surgem da interação do robô com o ambiente, despoletadas por eventos sensoriais. A abordagem proposta é testada numa estrutura de simulação dinâmica, sendo que várias experiências são efetuadas para avaliar os métodos e o desempenho dos mesmos.This work addresses the problem of learning to imitate human locomotion actions through low-level trajectories encoded with motion primitives and generalizing them to new situations from a single demonstration. In this line of thought, the main objectives of this work are twofold: The first is to analyze, extract and encode human demonstrations taken from motion capture data in order to model biped locomotion tasks. However, transferring motion skills from humans to robots is not limited to the simple reproduction, but requires the evaluation of their ability to adapt to new situations, as well as to deal with unexpected disturbances. Therefore, the second objective is to develop and evaluate a control framework for action shaping such that the single-demonstration can be modulated to varying situations, taking into account the dynamics of the robot and its environment. The idea behind the approach is to address the problem of generalization from a single-demonstration by combining two basic structures. The first structure is a pattern generator system consisting of movement primitives learned and modelled by dynamical systems (DS). This encoding approach possesses desirable properties that make them well-suited for trajectory generation, namely the possibility to change parameters online such as the amplitude and the frequency of the limit cycle and the intrinsic robustness against small perturbations. The second structure, which is embedded in the previous one, consists of coupled phase oscillators that organize actions into functional coordinated units. The changing contact conditions plus the associated impacts with the ground lead to models with multiple phases. Instead of forcing the robot’s motion into a predefined fixed timing, the proposed pattern generator explores transition between phases that emerge from the interaction of the robot system with the environment, triggered by sensor-driven events. The proposed approach is tested in a dynamics simulation framework and several experiments are conducted to validate the methods and to assess the performance of a humanoid robot

Locomoção de humanoides robusta e versátil baseada em controlo analítico e física residual

Humanoid robots are made to resemble humans but their locomotion abilities are far from ours in terms of agility and versatility. When humans walk on complex terrains or face external disturbances, they combine a set of strategies, unconsciously and efficiently, to regain stability. This thesis tackles the problem of developing a robust omnidirectional walking framework, which is able to generate versatile and agile locomotion on complex terrains. We designed and developed model-based and model-free walk engines and formulated the controllers using different approaches including classical and optimal control schemes and validated their performance through simulations and experiments. These frameworks have hierarchical structures that are composed of several layers. These layers are composed of several modules that are connected together to fade the complexity and increase the flexibility of the proposed frameworks. Additionally, they can be easily and quickly deployed on different platforms. Besides, we believe that using machine learning on top of analytical approaches is a key to open doors for humanoid robots to step out of laboratories. We proposed a tight coupling between analytical control and deep reinforcement learning. We augmented our analytical controller with reinforcement learning modules to learn how to regulate the walk engine parameters (planners and controllers) adaptively and generate residuals to adjust the robot’s target joint positions (residual physics). The effectiveness of the proposed frameworks was demonstrated and evaluated across a set of challenging simulation scenarios. The robot was able to generalize what it learned in one scenario, by displaying human-like locomotion skills in unforeseen circumstances, even in the presence of noise and external pushes.Os robôs humanoides são feitos para se parecerem com humanos, mas suas habilidades de locomoção estão longe das nossas em termos de agilidade e versatilidade. Quando os humanos caminham em terrenos complexos ou enfrentam distúrbios externos combinam diferentes estratégias, de forma inconsciente e eficiente, para recuperar a estabilidade. Esta tese aborda o problema de desenvolver um sistema robusto para andar de forma omnidirecional, capaz de gerar uma locomoção para robôs humanoides versátil e ágil em terrenos complexos. Projetámos e desenvolvemos motores de locomoção sem modelos e baseados em modelos. Formulámos os controladores usando diferentes abordagens, incluindo esquemas de controlo clássicos e ideais, e validámos o seu desempenho por meio de simulações e experiências reais. Estes frameworks têm estruturas hierárquicas compostas por várias camadas. Essas camadas são compostas por vários módulos que são conectados entre si para diminuir a complexidade e aumentar a flexibilidade dos frameworks propostos. Adicionalmente, o sistema pode ser implementado em diferentes plataformas de forma fácil. Acreditamos que o uso de aprendizagem automática sobre abordagens analíticas é a chave para abrir as portas para robôs humanoides saírem dos laboratórios. Propusemos um forte acoplamento entre controlo analítico e aprendizagem profunda por reforço. Expandimos o nosso controlador analítico com módulos de aprendizagem por reforço para aprender como regular os parâmetros do motor de caminhada (planeadores e controladores) de forma adaptativa e gerar resíduos para ajustar as posições das juntas alvo do robô (física residual). A eficácia das estruturas propostas foi demonstrada e avaliada em um conjunto de cenários de simulação desafiadores. O robô foi capaz de generalizar o que aprendeu em um cenário, exibindo habilidades de locomoção humanas em circunstâncias imprevistas, mesmo na presença de ruído e impulsos externos.Programa Doutoral em Informátic

Human inspired humanoid robots control architecture

This PhD Thesis tries to present a different point of view when talking about the development of control architectures for humanoid robots. Specifically, this Thesis is focused on studying the human postural control system as well as on the use of this knowledge to develop a novel architecture for postural control in humanoid robots. The research carried on in this thesis shows that there are two types of components for postural control: a reactive one, and other predictive or anticipatory. This work has focused on the development of the second component through the implementation of a predictive system complementing the reactive one. The anticipative control system has been analysed in the human case and it has been extrapolated to the architecture for controlling the humanoid robot TEO. In this way, its different components have been developed based on how humans work without forgetting the tasks it has been designed for. This control system is based on the composition of sensorial perceptions, the evaluation of stimulus through the use of the psychophysics theory of the surprise, and the creation of events that can be used for activating some reaction strategies (synergies) The control system developed in this Thesis, as well as the human being does, processes information coming from different sensorial sources. It also composes the named perceptions, which depend on the type of task the postural control acts over. The value of those perceptions is obtained using bio-inspired evaluation techniques of sensorial inference. Once the sensorial input has been obtained, it is necessary to process it in order to foresee possible disturbances that may provoke an incorrect performance of a task. The system developed in this Thesis evaluates the sensorial information, previously transformed into perceptions, through the use of the “Surprise Theory”, and it generates some events called “surprises” used for predicting the evolution of a task. Finally, the anticipative system for postural control can compose, if necessary, the proper reactions through the use of predefined movement patterns called synergies. Those reactions can complement or substitute completely the normal performance of a task. The performance of the anticipative system for postural control as well as the performance of each one of its components have been tested through simulations and the application of the results in the humanoid robot TEO from the RoboticsLab research group in the Systems Engineering and Automation Department from the Carlos III University of Madrid. ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Esta Tesis Doctoral pretende aportar un punto de vista diferente en el desarrollo de arquitecturas de control para robots humanoides. En concreto, esta Tesis se centra en el estudio del sistema de control postural humano y en la aplicación de este conocimiento en el desarrollo de una nueva arquitectura de control postural para robots humanoides. El estudio realizado en esta Tesis pone de manifiesto la existencia de una componente de control postural reactiva y otra predictiva o anticipativa. Este trabajo se ha centrado en el desarrollo de la segunda componente mediante la implementación de un sistema predictivo que complemente al sistema reactivo. El sistema de control anticipativo ha sido estudiado en el caso humano y extrapolado para la arquitectura de control del robot humanoide TEO. De este modo, sus diferentes componentes han sido desarrollados inspirándose en el funcionamiento humano y considerando las tareas para las que dicho robot ha sido concebido. Dicho sistema está basado en la composición de percepciones sensoriales, la evaluación de los estímulos mediante el uso de la teoría psicofísica de la sorpresa y la generación de eventos que sirvan para activar estrategias de reacción (sinergias). El sistema de control desarrollado en esta Tesis, al igual que el ser humano, procesa información de múltiples fuentes sensoriales y compone las denominadas percepciones, que dependen del tipo de tarea sobre la que actúa el control postural. El valor de estas percepciones es obtenido utilizando técnicas de evaluación bioinspiradas de inferencia sensorial. Una vez la entrada sensorial ha sido obtenida, es necesario procesarla para prever posibles perturbaciones que puedan ocasionar una incorrecta realización de una tarea. El sistema desarrollado en esta Tesis evalúa la información sensorial, previamente transformada en percepciones, mediante la ‘Teoría de la Sorpresa’ y genera eventos llamados ‘sorpresas’ que sirven para predecir la evolución de una tarea. Por último, el sistema anticipativo de control postural puede componer, si fuese necesario, las reacciones adecuadas mediante el uso de patrones de movimientos predefinidos llamados sinergias. Dichas reacciones pueden complementar o sustituir por completo la ejecución normal de una tarea. El funcionamiento del sistema anticipativo de control postural y de cada uno de sus componentes ha sido probado tanto por medio de simulaciones como por su aplicación en el robot humanoide TEO del grupo de investigación RoboticsLab en el Departamento de Ingeniería de Sistemas y Automática de la Universidad Carlos III de Madrid

Motion representation with spiking neural networks for grasping and manipulation

Die Natur bedient sich Millionen von Jahren der Evolution, um adaptive physikalische Systeme mit effizienten Steuerungsstrategien zu erzeugen. Im Gegensatz zur konventionellen Robotik plant der Mensch nicht einfach eine Bewegung und führt sie aus, sondern es gibt eine Kombination aus mehreren Regelkreisen, die zusammenarbeiten, um den Arm zu bewegen und ein Objekt mit der Hand zu greifen. Mit der Forschung an humanoiden und biologisch inspirierten Robotern werden komplexe kinematische Strukturen und komplizierte Aktor- und Sensorsysteme entwickelt. Diese Systeme sind schwierig zu steuern und zu programmieren, und die klassischen Methoden der Robotik können deren Stärken nicht immer optimal ausnutzen. Die neurowissenschaftliche Forschung hat große Fortschritte beim Verständnis der verschiedenen Gehirnregionen und ihrer entsprechenden Funktionen gemacht. Dennoch basieren die meisten Modelle auf groß angelegten Simulationen, die sich auf die Reproduktion der Konnektivität und der statistischen neuronalen Aktivität konzentrieren. Dies öffnet eine Lücke bei der Anwendung verschiedener Paradigmen, um Gehirnmechanismen und Lernprinzipien zu validieren und Funktionsmodelle zur Steuerung von Robotern zu entwickeln. Ein vielversprechendes Paradigma ist die ereignis-basierte Berechnung mit SNNs. SNNs fokussieren sich auf die biologischen Aspekte von Neuronen und replizieren deren Arbeitsweise. Sie sind für spike- basierte Kommunikation ausgelegt und ermöglichen die Erforschung von Mechanismen des Gehirns für das Lernen mittels neuronaler Plastizität. Spike-basierte Kommunikation nutzt hoch parallelisierten Hardware-Optimierungen mittels neuromorpher Chips, die einen geringen Energieverbrauch und schnelle lokale Operationen ermöglichen. In dieser Arbeit werden verschiedene SNNs zur Durchführung von Bewegungss- teuerung für Manipulations- und Greifaufgaben mit einem Roboterarm und einer anthropomorphen Hand vorgestellt. Diese basieren auf biologisch inspirierten funktionalen Modellen des menschlichen Gehirns. Ein Motor-Primitiv wird auf parametrische Weise mit einem Aktivierungsparameter und einer Abbildungsfunktion auf die Roboterkinematik übertragen. Die Topologie des SNNs spiegelt die kinematische Struktur des Roboters wider. Die Steuerung des Roboters erfolgt über das Joint Position Interface. Um komplexe Bewegungen und Verhaltensweisen modellieren zu können, werden die Primitive in verschiedenen Schichten einer Hierarchie angeordnet. Dies ermöglicht die Kombination und Parametrisierung der Primitiven und die Wiederverwendung von einfachen Primitiven für verschiedene Bewegungen. Es gibt verschiedene Aktivierungsmechanismen für den Parameter, der ein Motorprimitiv steuert — willkürliche, rhythmische und reflexartige. Außerdem bestehen verschiedene Möglichkeiten neue Motorprimitive entweder online oder offline zu lernen. Die Bewegung kann entweder als Funktion modelliert oder durch Imitation der menschlichen Ausführung gelernt werden. Die SNNs können in andere Steuerungssysteme integriert oder mit anderen SNNs kombiniert werden. Die Berechnung der inversen Kinematik oder die Validierung von Konfigurationen für die Planung ist nicht erforderlich, da der Motorprimitivraum nur durchführbare Bewegungen hat und keine ungültigen Konfigurationen enthält. Für die Evaluierung wurden folgende Szenarien betrachtet, das Zeigen auf verschiedene Ziele, das Verfolgen einer Trajektorie, das Ausführen von rhythmischen oder sich wiederholenden Bewegungen, das Ausführen von Reflexen und das Greifen von einfachen Objekten. Zusätzlich werden die Modelle des Arms und der Hand kombiniert und erweitert, um die mehrbeinige Fortbewegung als Anwendungsfall der Steuerungsarchitektur mit Motorprimitiven zu modellieren. Als Anwendungen für einen Arm (3 DoFs) wurden die Erzeugung von Zeigebewegungen und das perzeptionsgetriebene Erreichen von Zielen modelliert. Zur Erzeugung von Zeigebewegun- gen wurde ein Basisprimitiv, das auf den Mittelpunkt einer Ebene zeigt, offline mit vier Korrekturprimitiven kombiniert, die eine neue Trajektorie erzeugen. Für das wahrnehmungsgesteuerte Erreichen eines Ziels werden drei Primitive online kombiniert unter Verwendung eines Zielsignals. Als Anwendungen für eine Fünf-Finger-Hand (9 DoFs) wurden individuelle Finger-aktivierungen und Soft-Grasping mit nachgiebiger Steuerung modelliert. Die Greif- bewegungen werden mit Motor-Primitiven in einer Hierarchie modelliert, wobei die Finger-Primitive die Synergien zwischen den Gelenken und die Hand-Primitive die unterschiedlichen Affordanzen zur Koordination der Finger darstellen. Für jeden Finger werden zwei Reflexe hinzugefügt, zum Aktivieren oder Stoppen der Bewegung bei Kontakt und zum Aktivieren der nachgiebigen Steuerung. Dieser Ansatz bietet enorme Flexibilität, da Motorprimitive wiederverwendet, parametrisiert und auf unterschiedliche Weise kombiniert werden können. Neue Primitive können definiert oder gelernt werden. Ein wichtiger Aspekt dieser Arbeit ist, dass im Gegensatz zu Deep Learning und End-to-End-Lernmethoden, keine umfangreichen Datensätze benötigt werden, um neue Bewegungen zu lernen. Durch die Verwendung von Motorprimitiven kann der gleiche Modellierungsansatz für verschiedene Roboter verwendet werden, indem die Abbildung der Primitive auf die Roboterkinematik neu definiert wird. Die Experimente zeigen, dass durch Motor- primitive die Motorsteuerung für die Manipulation, das Greifen und die Lokomotion vereinfacht werden kann. SNNs für Robotikanwendungen ist immer noch ein Diskussionspunkt. Es gibt keinen State-of-the-Art-Lernalgorithmus, es gibt kein Framework ähnlich dem für Deep Learning, und die Parametrisierung von SNNs ist eine Kunst. Nichtsdestotrotz können Robotikanwendungen - wie Manipulation und Greifen - Benchmarks und realistische Szenarien liefern, um neurowissenschaftliche Modelle zu validieren. Außerdem kann die Robotik die Möglichkeiten der ereignis- basierten Berechnung mit SNNs und neuromorpher Hardware nutzen. Die physikalis- che Nachbildung eines biologischen Systems, das vollständig mit SNNs implementiert und auf echten Robotern evaluiert wurde, kann neue Erkenntnisse darüber liefern, wie der Mensch die Motorsteuerung und Sensorverarbeitung durchführt und wie diese in der Robotik angewendet werden können. Modellfreie Bewegungssteuerungen, inspiriert von den Mechanismen des menschlichen Gehirns, können die Programmierung von Robotern verbessern, indem sie die Steuerung adaptiver und flexibler machen

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion

Push Recovery Through Walking Phase Modification for Bipedal Locomotion

Ph.DDOCTOR OF PHILOSOPH

Pattern Generation for Rough Terrain Locomotion with Quadrupedal Robots:Morphed Oscillators & Sensory Feedback

Animals are able to locomote on rough terrain without any apparent difficulty, but this does not mean that the locomotor system is simple. The locomotor system is actually a complex multi-input multi-output closed-loop control system. This thesis is dedicated to the design of controllers for rough terrain locomotion, for animal-like quadrupedal robots. We choose the problem of blind rough terrain locomotion as the target of experiments. Blind rough terrain locomotion requires continuous and momentary corrections of leg movements and body posture, and provides a proper testbed to observe the interaction of different mod- ules involved in locomotion control. As for the specific case of this thesis, we have to design rough terrain locomotion controllers that do not depend on the torque-control capability, have limited sensing, and have to be computationally light, all due to the properties of the robotics platform that we use. We propose that a robust locomotion controller, taking into account the aforementioned constraints, is constructed from at least three modules: 1) pattern generators providing the nominal patterns of locomotion; 2) A posture controller continuously adjusting the attitude of the body and keeping the robot upright; and 3) quick reflexes to react to unwanted momentary events like stumbling or an external force impulse. We introduce the framework of morphed oscillators to systematize the design of pattern gen- erators realized as coupled nonlinear oscillators. Morphed oscillators are nonlinear oscillators that can encode arbitrary limit cycle shapes and simultaneously have infinitely large basins of attraction. More importantly, they provide dynamical systems that can assume the role of feedforward locomotion controllers known as Central Pattern Generators (CPGs), and accept discontinuous sensory feedback without the risk of producing discontinuous output. On top of the CPG module, we add a kinematic model-based posture controller inspired by virtual model control (VMC), to control the body attitude. Virtual model control produces forces, and through the application of the Jacobian transpose method, generates torques which are added to the CPG torques. However, because our robots do not have a torque- control capability, we adapt the posture controller by producing task-space velocities instead of forces, thus generating joint-space velocity feedback signals. Since the CPG model used for locomotion generates joint velocities and accepts feedback without the fear of instability or discontinuity, the posture control feedback is easily integrated into the CPG dynamics. More- over, we introduce feedback signals for adjusting the posture by shifting the trunk positions, which directly update the limit cycle shape of the morphed oscillator nodes of the CPG. Reflexes are added, with minimal complexity, to react to momentary events. We implement simple impulse-based feedback mechanisms inspired by animals and successful rough terrain robots to 1) flex the leg if the robot is stumbling (stumbling correction reflex); 2) extend the leg if an expected contact is missing (leg extension reflex); or 3) initiate a lateral stepping sequence in response to a lateral external perturbation. CPG, posture controller, and reflexes are put together in a modular control architecture alongside additional modules that estimate inclination, control speed and direction, maintain timing of feedback signals, etc. [...