Multiple chaotic central pattern generators with learning for legged locomotion and malfunction compensation

An originally chaotic system can be controlled into various periodic dynamics. When it is implemented into a legged robot's locomotion control as a central pattern generator (CPG), sophisticated gait patterns arise so that the robot can perform various walking behaviors. However, such a single chaotic CPG controller has difficulties dealing with leg malfunction. Specifically, in the scenarios presented here, its movement permanently deviates from the desired trajectory. To address this problem, we extend the single chaotic CPG to multiple CPGs with learning. The learning mechanism is based on a simulated annealing algorithm. In a normal situation, the CPGs synchronize and their dynamics are identical. With leg malfunction or disability, the CPGs lose synchronization leading to independent dynamics. In this case, the learning mechanism is applied to automatically adjust the remaining legs' oscillation frequencies so that the robot adapts its locomotion to deal with the malfunction. As a consequence, the trajectory produced by the multiple chaotic CPGs resembles the original trajectory far better than the one produced by only a single CPG. The performance of the system is evaluated first in a physical simulation of a quadruped as well as a hexapod robot and finally in a real six-legged walking machine called AMOSII. The experimental results presented here reveal that using multiple CPGs with learning is an effective approach for adaptive locomotion generation where, for instance, different body parts have to perform independent movements for malfunction compensation.Comment: 48 pages, 16 figures, Information Sciences 201

Technical Report: A Contact-aware Feedback CPG System for Learning-based Locomotion Control in a Soft Snake Robot

Integrating contact-awareness into a soft snake robot and efficiently controlling its locomotion in response to contact information present significant challenges. This paper aims to solve contact-aware locomotion problem of a soft snake robot through developing bio-inspired contact-aware locomotion controllers. To provide effective contact information for the controllers, we develop a scale covered sensor structure mimicking natural snakes' \textit{scale sensilla}. In the design of control framework, our core contribution is the development of a novel sensory feedback mechanism of the Matsuoka central pattern generator (CPG) network. This mechanism allows the Matsuoka CPG system to work like a "spine cord" in the whole contact-aware control scheme, which simultaneously takes the stimuli including tonic input signals from the "brain" (a goal-tracking locomotion controller) and sensory feedback signals from the "reflex arc" (the contact reactive controller), and generate rhythmic signals to effectively actuate the soft snake robot to slither through densely allocated obstacles. In the design of the "reflex arc", we develop two types of reactive controllers -- 1) a reinforcement learning (RL) sensor regulator that learns to manipulate the sensory feedback inputs of the CPG system, and 2) a local reflexive sensor-CPG network that directly connects sensor readings and the CPG's feedback inputs in a special topology. These two reactive controllers respectively facilitate two different contact-aware locomotion control schemes. The two control schemes are tested and evaluated in the soft snake robot, showing promising performance in the contact-aware locomotion tasks. The experimental results also further verify the benefit of Matsuoka CPG system in bio-inspired robot controller design.Comment: 17 pages, 19 figure

Controlling swimming and crawling in a fish robot using a central pattern generator

Online trajectory generation for robots with multiple degrees of freedom is still a difficult and unsolved problem, in particular for non-steady state locomotion, that is, when the robot has to move in a complex environment with continuous variations of the speed, direction, and type of locomotor behavior. In this article we address the problem of controlling the non-steady state swimming and crawling of a novel fish robot. For this, we have designed a control architecture based on a central pattern generator (CPG) implemented as a system of coupled nonlinear oscillators. The CPG, like its biological counterpart, can produce coordinated patterns of rhythmic activity while being modulated by simple control parameters. To test our controller, we designed BoxyBot, a simple fish robot with three actuated fins capable of swimming in water and crawling on firm ground. Using the CPG model, the robot is capable of performing and switching between a variety of different locomotor behaviors such as swimming forwards, swimming backwards, turning, rolling, moving upwards/downwards, and crawling. These behaviors are triggered and modulated by sensory input provided by light, water, and touch sensors. Results are presented demonstrating the agility of the robot and interesting properties of a CPG-based control approach such as stability of the rhythmic patterns due to limit cycle behavior, and the production of smooth trajectories despite abrupt changes of control parameters. The robot is currently used in a temporary 20-month long exhibition at the EPFL. We present the hardware setup that was designed for the exhibition, and the type of interactions with the control system that allow visitors to influence the behavior of the robot. The exhibition is useful to test the robustness of the robot for long term use, and to demonstrate the suitability of the CPG-based approach for interactive control with a human in the loop. This article is an extended version of an article presented at BioRob2006 the first IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronic

Reinforcement Learning of CPG-regulated Locomotion Controller for a Soft Snake Robot

Intelligent control of soft robots is challenging due to the nonlinear and difficult-to-model dynamics. One promising model-free approach for soft robot control is reinforcement learning (RL). However, model-free RL methods tend to be computationally expensive and data-inefficient and may not yield natural and smooth locomotion patterns for soft robots. In this work, we develop a bio-inspired design of a learning-based goal-tracking controller for a soft snake robot. The controller is composed of two modules: An RL module for learning goal-tracking behaviors given the unmodeled and stochastic dynamics of the robot, and a central pattern generator (CPG) with the Matsuoka oscillators for generating stable and diverse locomotion patterns. We theoretically investigate the maneuverability of Matsuoka CPG's oscillation bias, frequency, and amplitude for steering control, velocity control, and sim-to-real adaptation of the soft snake robot. Based on this analysis, we proposed a composition of RL and CPG modules such that the RL module regulates the tonic inputs to the CPG system given state feedback from the robot, and the output of the CPG module is then transformed into pressure inputs to pneumatic actuators of the soft snake robot. This design allows the RL agent to naturally learn to entrain the desired locomotion patterns determined by the CPG maneuverability. We validated the optimality and robustness of the control design in both simulation and real experiments, and performed extensive comparisons with state-of-art RL methods to demonstrate the benefit of our bio-inspired control design.Comment: 20 pages, 17 figures, 4 tables, in IEEE Transactions on Robotic

Design and control of amphibious robots with multiple degrees of freedom

This thesis presents the design and realization of two generations of robot elements that can be assembled together to construct amphibious mobile robots. These elements, designed to be individually waterproof and having their own battery, motor controller, and motor, have been used to actually construct a snake, a boxfish and a salamander robot. Central pattern generator (CPG) models inspired from those found in vertebrates have been used for online trajectory generation on these robots and implemented on their onboard locomotion controllers. CPGs proved to be an interesting way of controlling complex robots, providing a simple interface which hides the complexity of the robot to the end user. Online learning algorithms that can be used to dynamically adapt the locomotion parameters to the environment have been implemented. Finally, this work also shows how robotics can be a useful tool to verify biological hypotheses. For instance, the salamander robot has been used to test a model of CPG for salamander locomotion