Fast biped walking with a neuronal controller and physical computation

Biped walking remains a difficult problem and robot models can greatly {facilitate} our understanding of the underlying biomechanical principles as well as their neuronal control. The goal of this study is to specifically demonstrate that stable biped walking can be achieved by combining the physical properties of the walking robot with a small, reflex-based neuronal network, which is governed mainly by local sensor signals. This study shows that human-like gaits emerge without {specific} position or trajectory control and that the walker is able to compensate small disturbances through its own dynamical properties. The reflexive controller used here has the following characteristics, which are different from earlier approaches: (1) Control is mainly local. Hence, it uses only two signals (AEA=Anterior Extreme Angle and GC=Ground Contact) which operate at the inter-joint level. All other signals operate only at single joints. (2) Neither position control nor trajectory tracking control is used. Instead, the approximate nature of the local reflexes on each joint allows the robot mechanics itself (e.g., its passive dynamics) to contribute substantially to the overall gait trajectory computation. (3) The motor control scheme used in the local reflexes of our robot is more straightforward and has more biological plausibility than that of other robots, because the outputs of the motorneurons in our reflexive controller are directly driving the motors of the joints, rather than working as references for position or velocity control. As a consequence, the neural controller and the robot mechanics are closely coupled as a neuro-mechanical system and this study emphasises that dynamically stable biped walking gaits emerge from the coupling between neural computation and physical computation. This is demonstrated by different walking experiments using two real robot as well as by a Poincar\'{e} map analysis applied on a model of the robot in order to assess its stability. In addition, this neuronal control structure allows the use of a policy gradient reinforcement learning algorithm to tune the parameters of the neurons in real-time, during walking. This way the robot can reach a record-breaking walking speed of 3.5 leg-lengths per second after only a few minutes of online learning, which is even comparable to the fastest relative speed of human walking

Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion

Learning policies for bipedal locomotion can be difficult, as experiments are expensive and simulation does not usually transfer well to hardware. To counter this, we need al- gorithms that are sample efficient and inherently safe. Bayesian Optimization is a powerful sample-efficient tool for optimizing non-convex black-box functions. However, its performance can degrade in higher dimensions. We develop a distance metric for bipedal locomotion that enhances the sample-efficiency of Bayesian Optimization and use it to train a 16 dimensional neuromuscular model for planar walking. This distance metric reflects some basic gait features of healthy walking and helps us quickly eliminate a majority of unstable controllers. With our approach we can learn policies for walking in less than 100 trials for a range of challenging settings. In simulation, we show results on two different costs and on various terrains including rough ground and ramps, sloping upwards and downwards. We also perturb our models with unknown inertial disturbances analogous with differences between simulation and hardware. These results are promising, as they indicate that this method can potentially be used to learn control policies on hardware.Comment: To appear in International Conference on Humanoid Robots (Humanoids '2016), IEEE-RAS. (Rika Antonova and Akshara Rai contributed equally

Using evolutionary artificial neural networks to design hierarchical animat nervous systems.

The research presented in this thesis examines the area of control systems for robots or animats (animal-like robots). Existing systems have problems in that they require a great deal of manual design or are limited to performing jobs of a single type. For these reasons, a better solution is desired. The system studied here is an Artificial Nervous System (ANS) which is biologically inspired; it is arranged as a hierarchy of layers containing modules operating in parallel. The ANS model has been developed to be flexible, scalable, extensible and modular. The ANS can be implemented using any suitable technology, for many different environments. The implementation focused on the two lowest layers (the reflex and action layers) of the ANS, which are concerned with control and rhythmic movement. Both layers were realised as Artificial Neural Networks (ANN) which were created using Evolutionary Algorithms (EAs). The task of the reflex layer was to control the position of an actuator (such as linear actuators or D.C. motors). The action layer performed the task of Central Pattern Generators (CPG), which produce rhythmic patterns of activity. In particular, different biped and quadruped gait patterns were created. An original neural model was specifically developed for assisting in the creation of these time-based patterns. It is shown in the thesis that Artificial Reflexes and CPGs can be configured successfully using this technique. The Artificial Reflexes were better at generalising across different actuators, without changes, than traditional controllers. Gaits such as pace, trot, gallop and pronk were successfully created using the CPGs. Experiments were conducted to determine whether modularity in the networks had an impact. It has been demonstrated that the degree of modularization in the network influences its evolvability, with more modular networks evolving more efficiently

The Benefit of Combining Neuronal Feedback and Feed-Forward Control for Robustness in Step Down Perturbations of Simulated Human Walking Depends on the Muscle Function

It is often assumed that the spinal control of human locomotion combines feed-forward central pattern generation with sensory feedback via muscle reflexes. However, the actual contribution of each component to the generation and stabilization of gait is not well understood, as direct experimental evidence for either is difficult to obtain. We here investigate the relative contribution of the two components to gait stability in a simulation model of human walking. Specifically, we hypothesize that a simple linear combination of feedback and feed-forward control at the level of the spinal cord improves the reaction to unexpected step down perturbations. In previous work, we found preliminary evidence supporting this hypothesis when studying a very reduced model of rebounding behaviors. In the present work, we investigate if the evidence extends to a more realistic model of human walking. We revisit a model that has previously been published and relies on spinal feedback control to generate walking. We extend the control of this model with a feed-forward muscle activation pattern. The feed-forward pattern is recorded from the unperturbed feedback control output. We find that the improvement in the robustness of the walking model with respect to step down perturbations depends on the ratio between the two strategies and on the muscle to which they are applied. The results suggest that combining feed-forward and feedback control is not guaranteed to improve locomotion, as the beneficial effects are dependent on the muscle and its function during walking

A Simple 3D-Only Evolutionary Bipedal System with Albatross Morphology for Increased Performance

Bipedal walking is a difficult behaviour to encode into an evolutionary neural network, particularly in three-dimensional environments. Agents must be constantly maintaining balance alongside their primary objectives. Here we re-implement a simple evolutionary bipedal system, achieving high fitness and stepping gaits in 3D without the preliminary 2D bootstrapping process required by the original work. This high-performing system, with its deliberately simple neurocontroller, provides an excellent foundation for the community to use for the evolution or learning of more complex behaviours in bipeds. We also investigate the effects of modified morphology with the system, significantly improving agent fitness by evolving networks alongside morphologies resembling a baby albatross. The agents with albatross morphologies travel up to three times further than default agents. We then test incrementally evolving agent morphology via the simultaneous evolution of a separate morphological genotype. We initialised this genotype either alongside a high-performing controller or from a completely random point in both fitness landscapes. Agents evolved from this random initialisation travel up to four times further than default agents. One randomly initialised incremental morphology also achieves gaits with significantly higher upper body and swing knee controller input weights than the default

Stabilization of bipedal walking based on compliance control

The embodiment of physical compliance in humanoid robots, inspired by biology, improves the robustness of locomotion in unknown environments. The mechanical implementation using elastic materials demands a further combination together with controlled compliance to make the intrinsic compliance more effective. We hereby present an active compliance control to stabilize the humanoid robots for standing and walking tasks. Our actively controlled compliance is achieved via admittance control using closed-loop feedback of the six axis force/torque sensors in the feet. The modeling and theoretical formulation are presented, followed by the simulation study. Further, the control algorithms were validated on a real humanoid robot COMAN with inherent compliance. A series of experimental comparisons were studied, including standing balancing against impacts, straight walking, and omni-directional walking, to demonstrate the necessity and the effectiveness of applying controlled compliance on the basis of physical elasticity to enhance compliant foot-ground interaction for the successful locomotion. All data from simulations and experiments related with the proposed controller and the performance are presented, analyzed, and discussed

Development of a Neuromechanical Model for Investigating Sensorimotor Interactions During Locomotion

Recently it has been suggested that the use of neuromechanical simulations could be used to further our understanding of the neural control mechanisms involved in the control of animal locomotion. The models used to carry out these neuromechanical simulations typically consist of a representation of the neural control systems involved in walking and a representation of the mechanical locomotor apparatus. These separate models are then integrated to produce motion of the locomotor apparatus based on signals that are generated by the neural control models. Typically in past neuromechanical simulations of human walking the parameters of the neural control model have been specifically chosen to produce a walking pattern that resembles the normal human walking pattern as closely as possible. Relatively few of these studies have systematically tested the effect of manipulating the control parameters on the walking pattern that is produced by the locomotor apparatus. The goal of this thesis was to develop models of the locomotor control system and the human locomotor apparatus and systematically manipulate several parameters of the neural control system and determine what effects these parameters would have on the walking pattern of the mechanical model. Specifically neural control models were created of the Central Pattern Generator (CPG), feedback mechanisms from muscle spindles and contact sensors that detect when the foot was contact with the ground. Two models of the human locomotor apparatus were used to evaluate the outputs of the neural control systems; the first was a rod pendulum, which represented a swinging lower-limb, while the second was a 5-segment biped model, which included contact dynamics with the ground and a support system model to maintain balance. The first study of this thesis tested the ability of a CPG model to control the frequency and amplitude of the pendulum model of the lower-limb, with a strictly feedforward control mechanism. It was found that the frequency of the pendulum’s motion was directly linked (or entrained) to the frequency of the CPG’s output. It was also found that the amplitude of the pendulum’s motion was affected by the frequency of the CPG’s output, with the greatest amplitude of motion occurring when the frequency of the CPG matched the pendulum’s natural frequency. The effects of altering several other parameters of the pendulum model, such as the initial angle, the magnitude of the applied viscous damping or the moment arms of the muscles, were also analyzed. The second study again used the pendulum model, and added feedback to the neural control model, via output from simulated muscle spindles. The output from these spindle models was used to trigger a simulated stretch reflex. It was found that the addition of feedback led to sensory entrainment of the CPG output to the natural frequency of the pendulum. The effects of altering the muscle spindle’s sensitivity to length and velocity changes were also examined. The ability of this type of feedback system to respond to mechanical perturbations was also analyzed. The third and fourth studies used a biped model of the musculoskeletal system to assess the effects of altering the parameters of the neural control systems that were developed in the first two studies. In the third study, the neural control system consisted only of feedforward control from the CPG model. It was found that the walking speed of the biped model could be controlled by altering the frequency of the CPG’s output. It was also observed that variability of the walking pattern was decreased when there was a moderate level of inhibition between the CPGs of the left and right hip joints. The final study added feedback from muscle receptors and from contact sensors with the ground. It was found that the most important source of feedback was from the contact sensors to the extensor centres of the CPG. This feedback increased the level of extensor activity and produced significantly faster walking speeds when compared to other types of feedback. This thesis was successful in testing the effects of several control parameters of the neural control system on the movement of mechanical systems. Particularly important findings included the importance of connectivity between the CPGs of the left and right hip joints and positive feedback regarding the loading of the limb for establishing an appropriate forward walking speed. It is hoped that the models developed in this thesis can form the basis of future neuromechanical models and that the simulations carried out in this thesis help provide a better understanding of the interactions between neural and mechanical systems during the control of locomotion