Evolution of central pattern generators for the control of a five-link bipedal walking mechanism

Central pattern generators (CPGs), with a basis is neurophysiological studies, are a type of neural network for the generation of rhythmic motion. While CPGs are being increasingly used in robot control, most applications are hand-tuned for a specific task and it is acknowledged in the field that generic methods and design principles for creating individual networks for a given task are lacking. This study presents an approach where the connectivity and oscillatory parameters of a CPG network are determined by an evolutionary algorithm with fitness evaluations in a realistic simulation with accurate physics. We apply this technique to a five-link planar walking mechanism to demonstrate its feasibility and performance. In addition, to see whether results from simulation can be acceptably transferred to real robot hardware, the best evolved CPG network is also tested on a real mechanism. Our results also confirm that the biologically inspired CPG model is well suited for legged locomotion, since a diverse manifestation of networks have been observed to succeed in fitness simulations during evolution.Comment: 11 pages, 9 figures; substantial revision of content, organization, and quantitative result

Matsuoka's CPG With Desired Rhythmic Signals for Adaptive Walking of Humanoid Robots

The desired rhythmic signals for adaptive walking of humanoid robots should have proper frequencies, phases, and shapes. Matsuoka's central pattern generator (CPG) is able to generate rhythmic signals with reasonable frequencies and phases, and thus has been widely applied to control the movements of legged robots, such as walking of humanoid robots. However, it is difficult for this kind of CPG to generate rhythmic signals with desired shapes, which limits the adaptability of walking of humanoid robots in various environments. To address this issue, a new framework that can generate desired rhythmic signals for Matsuoka's CPG is presented. The proposed framework includes three main parts. First, feature processing is conducted to transform the Matsuoka's CPG outputs into a normalized limit cycle. Second, by combining the normalized limit cycle with robot feedback as the feature inputs and setting the required learning objective, the neural network (NN) learns to generate desired rhythmic signals. Finally, in order to ensure the continuity of the desired rhythmic signals, signal filtering is applied to the outputs of NN, with the aim of smoothing the discontinuous parts. Numerical experiments on the proposed framework suggest that it can not only generate a variety of rhythmic signals with desired shapes but also preserve the frequency and phase properties of Matsuoka's CPG. In addition, the proposed framework is embedded into a control system for adaptive omnidirectional walking of humanoid robot NAO. Extensive simulation and real experiments on this control system demonstrate that the proposed framework is able to generate desired rhythmic signals for adaptive walking of NAO on fixed and changing inclined surfaces. Furthermore, the comparison studies verify that the proposed framework can significantly improve the adaptability of NAO's walking compared with the other methods

Gait-Behavior Optimization Considering Arm Swing and Toe Mechanisms for Biped Robot on Rough Road

芝浦工業大学2019年

A review of gait optimization based on evolutionary computation

Gait generation is very important as it directly affects the quality of locomotion of legged robots. As this is an optimization problem with constraints, it readily lends itself to Evolutionary Computation methods and solutions. This paper reviews the techniques used in evolution-based gait optimization, including why Evolutionary Computation techniques should be used, how fitness functions should be composed, and the selection of genetic operators and control parameters. This paper also addresses further possible improvements in the efficiency and quality of evolutionary gait optimization, some problems that have not yet been resolved and the perspectives for related future research

Intelligent System Synthesis for Dynamic Locomotion Behavior in Multi-legged Robots

Robot technology has been implemented in many fields of our life, such as entertainment, security, rescue, rehabilitation, social life, the military, and etc. Multi-legged robot always exist in many fields, therefore it is important to be developed. Motion capabilities of the robot will be a main focus to be developed. Current development or conventional model of motion capabilities have several issues in saturation of development. There are some limitation in dynamic factors such as, locomotion generator, flexibility of motion planning, and smoothness of movement. Therefore, in this research, natural based computation are implemented as the basic model. There are three subsystems to be developed and integrated, (1) locomotion behavior model, (2) stability behavior model, and (3) motion planning model. Since individual people has different walking behavior in each walking direction and walking speed, locomotion behavior learning model of omni-directional bio-inspired locomotion which is generating different walking behavior in different walking provision are required to be developed. Step length in sagital and coronal direction, and degree of turning are considered parameters in walking provision. In proposed omni-directional walking model, interconnection structures composed by 16 neurons where 1 leg is represented by 4 joints and 1 joint is represented by 2 motor neurons. In order to acquire walking behavior in certain walking provision, the interconnection structure is optimized by multi-objectives evolutionary algorithm. For acquiring the diversity of references, several optimized interconnection structures are generated in optimization processes in different walking provisions. Learning models are proposed for solving non-linearity of relationship between walking input and walking output representing the synaptic weight of interconnection structure, where one learning model representing one walking parameter. Furthermore, by using optimized model, walking behavior can be generated with unsealed walking provision. Smooth walking transition with low error of desired walking provision was proved based on several numerical experiments in physical computer simulation. In stability behavior model, neuro-based push recovery controller is applied in multi-legged robot in order to keep the stability with minimum energy required. There are three motion patterns in individual people behavior when it gets external perturbation, those are ankle behavior, hip behavior, and step behavior. We propose a new model of Modular Recurrent Neural Network (MRNN) for performing online learning system in each motion behavior. MRNN consists of several recurrent neural networks (RNNs) working alternatively depending on the condition. MRNN performs online learning process of each motion behavior controller independently. The aim of push recovery controller is to manage the motion behavior controller by minimizing the energy required for responding to the external perturbation. This controller selects the appropriate motion behavior and adjusts the gain that represent the influence of the motion behavior to certain push disturbance based on behavior graphs which is generated by adaptive regression spline. We applied the proposed controller to the humanoid robot that has small footprint in open dynamics engine. Experimental result shows the effectiveness of the push controller stabilizing the external perturbation with minimum energy required. Proposed motion planning model presents a natural mechanism of the human brain for generating a dynamic path planning in 3-D rough terrain. The proposed model not only emphasizes the inner state process of the neuron but also the development process of the neurons in the brain. There are two information transmission processes in this proposed model, the forward transmission activity for constructing the neuron connections to find the possible way and the synaptic pruning activity with backward neuron transmission for finding the best pathway from current position to target position and reducing inefficient neuron with its synaptic connections. In order to respond and avoid the unpredictable obstacle, dynamic path planning is also considered in this proposed model. An integrated system for applying the proposed model in the actual experiments is also presented. In order to confirm the effectiveness of the proposed model, we applied the integrated system in the pathway of a four-legged robot on rough terrain in computer simulation. For analyzing and proving the flexibility of proposed model, unpredictable collision is also performed in those experiments. The model can find the best pathway and facilitate the safe movement of the robot. When the robot found an unpredictable collision, the path planner dynamically changed the pathway. The proposed path planning model is capable to be applied in further advance implementation. In order to implement the motion capabilities in real cases, all subsystem should be integrated into one interconnected motion capabilities model. We applied small quadruped robot equipped with IMU, touch sensor, and dual ultrasonic sensor for performing motion planning in real terrain from starting point to goal point. Before implemented, topological map is generated by Kinect camera. In this implementation, all subsystem were analyzed and performed well and the robot able to stop in the goal point. These implementation proved the effectiveness of the system integration, the motion planning model is able to generate safe path planning, the locomotion model is able to generate flexible movement depending on the walking provision from motion planning model, and the stability model can stabilize the robot on rough terrain. Generally, the proposed model can be expected to bring a great contribution to the motion capabilities development and can be used as alternative model for acquiring the dynamism and efficient model in the future instead of conventional model usage. In the future, the proposed model can be applied into any legged robot as navigation, supporter, or rescue robot in unstable environmental condition. In addition, we will realize a cognitive locomotion that generates multiple gaits depending on the 3 aspects, embodiment, locomotion generator, and cognition model. A dynamic neuro-locomotion integrated with internal and external sensory information for correlating with the environmental condition will be designed.ロボット技術は、エンターテイメント、セキュリティ、救助、リハビリ、社会生活、軍事などの様々な生活分野に実現さている。多脚ロポットは常に多くの分野に存在するため開発することが重要である。ロボットの運動能力が開発の主要となっている。現状の開発されている動作能力は,飽和状態にある。いくつかの動的な要因により、歩行生成器、動作計画の柔軟性、および動作の滑らかさ等に制限がある。そこで、本研究では、基本的なモデルとして自然計算に基づく方法論を実装する、また、本研究では、歩行動作モデル、安定動作モデル、や運動計画モデルからなる3つのサブシステムを開発し統合する。人間は歩行方向と速度に応じて歩行動作が異なるため、異なる歩行軸では異なる歩行動作を生成するという全方位生物的な運動の歩行動作学習モデルが開発には要求される。球欠および制御方向のステップ長や旋回の度合いは,歩行軸のパラメータとして考慮される。提案した全方位歩行モデルでは,1肢につき16個のニューロンによって構成される相互接続構造を4つの関節によって表現する。また、1つの関節は,2個のモータニューロンによって表現する。一定の歩行軸での歩行動作を獲得するために,本研究では,多目的進化アルゴリズムによって最適化を行う。提案手法では、参照点の多様性を獲得するために,異なる歩行軸においていくつかの最適な相互接続構造が生成される。相互接続構造のシナプス重みを表現している歩行入力と出力間の非線形な関係を解くための学習モデルを構築する。本手法では,1つの学習モデルが1つの歩行パラメータで表現され、最適化されたモデルを用いることにより,歩行動作は,スケーリングされていない歩行軸を生成することが可能となる,物理演算シミュレーションを用いた実験により,誤差の少ない歩行軸の滑らかな歩行遷移を本実験では示している。安定動作モデルでは、必要最小限のエネルギーで安定性を維持するため多足歩行ロボットにニューロベースプッシュリカバリ制御器を適用した。外力をを受けたとき,人間の行動には足首の動作・股関節の動作・踏み動作の3つの動作パターンが存在する。本研究では,各運動動作におけるオンライン学習システムを実現するために、モジュラーリカレントニューラルネットワーク(MRNN)を用いた新たな学習モデルを提案する。MRNNは状況に応じて選択される複数のリカレントニューラルネットワーク(RNN)によって構成される。MRNNは各運動動作コントローラのオンライン学習プロセスを独立して実行する。プッシュリカバリ制御器の目的は、外乱に応じてエネルギー最小化を行うことによって運動動作制御器を管理することである。この制御器は適切な運動動作を選択し,適応回帰スプラインにより生成された動作グラフに基づき押し動作に対して最も影響を及ぼす運動動作のゲインの調整を行う。提案した制御器をOpen Dynamics Engine(ODE)上で小さな足の長さを持つヒューマノイドロボットに適用し,必要最小限のエネルギーで外力に対して安定させるプッシュリカバリ制御器の有効性を示している。3次元の不整地における動的な経路計画を生成するために,人間の自然な脳機能に基づいた動作計画手法を提案する。本モデルは、ニューロンの内部状態過程だけでなく、脳内のニューロンの発達過程も重視している。本モデルは二つのアルゴリズムに構成される。1つは、通過可能な道を見つけるために構築される接続的なニューロン活動である順方向伝達活動であり,もう1つは、現在位置から最適経路を見つけるために、シナプス結合を用いて非効率的なニューロンを減少させる逆方向にニューロン伝達を行うシナプスプルーニング活動である。また,予測不可能な衝突を回避するために,動的な経路計画も実行される。さらに、実環境において提案されたモデルを実現するための統合システムも提示される。提案モデルの有効性を検証するために,コンピュータシミュレーション上で、不整地環境の4足歩行ロボットに関するシミュレーション環境を実装した。これらの実験では,予測不能な衝突に関する実験も行った。本モデルは、最適経路を見つけ出しロボットの安全な移動を実現できた。さらに、ロボットが予測できない衝突を検出した場合,経路計画アルゴリズムが経路を動的に変更可能であることを示している。これらのことから、提案された経路計画モデルはさらなる先進的な展開が実現可能であると考えられる。実環境における運動能力を実装するためには、すべてのサブシステムを1つの運動能力モデルに統合する必要がある。そこで本研究では、IMU、タッチセンサ、2つの超音波センサを搭載した小型の4足歩行ロポットを用いた実環境において出発地点から目的地点までの運動計画を行った、本実装では、3次元距離計測センサであるKinecを用い3次元空間の位相構造を生成する。また、本実装では、すべてのサブシステムが分析され、ロボットは目的地点で停止することができた。さらに、安全な経路計画を生成することができたことからシステム統合の有効性が確認できた。また、歩行モデルにより歩行軸に応じた柔軟な動きが生成されることで、この安定性モデルは不整地環撹でもロボットの歩行を安定させることができた。これらのことから、本提案モデルは運動能力への多大な貢献が期待され、ダイナミクスを獲得するための代替モデルとして使用することができ,現在よく使用されているモデルに代わる効率的なモデルとなることが考えられる。今後の課題としては,不安定な環境下におけるナビゲーション・支援・レスキューロボットといった任意の肢の数を持つ多足歩行ロボットへの本提案モデルの適用があげられる。さらに,身体性,歩行生成,認知モデルの3つの観点から複数の歩容を生成する認知的歩行を実現することを考えている。環境と相互作用するためのモデルとして、内界センサと外界センサ情報を統合した動的ニューロ歩行を実現する予定である。首都大学東京, 2018-03-25, 修士（工学）首都大学東

Using Reinforcement Learning in the tuning of Central Pattern Generators

Dissertação de mestrado em Engenharia InformáticaÉ objetivo deste trabalho aplicar técnicas de Reinforcement Learning em tarefas de aprendizagem e locomoção de robôs. Reinforcement Learning é uma técnica de aprendizagem útil no que diz respeito à locomoção de robôs, devido à ênfase que dá à interação direta entre o agente e o meio ambiente, e ao facto de não exigir supervisão ou modelos completos, ao contrário do que acontece nas abordagens clássicas. O objetivo desta técnica consiste na decisão das ações a tomar, de forma a maximizar uma recompensa cumulativa, tendo em conta o facto de que as decisões podem afetar não só as recompensas imediatas, como também as futuras. Neste trabalho será apresentada a estrutura e funcionamento do Reinforcement Learning e a sua aplicação em Central Pattern Generators, com o objetivo de gerar locomoção adaptativa otimizada. De forma a investigar e identificar os pontos fortes e capacidades do Reinforcement Learning, e para demonstrar de uma forma simples este tipo de algoritmos, foram implementados dois casos de estudo baseados no estado da arte. No que diz respeito ao objetivo principal desta tese, duas soluções diferentes foram abordadas: uma primeira baseada em métodos Natural-Actor Critic, e a segunda, em Cross-Entropy Method. Este último algoritmo provou ser capaz de lidar com a integração das duas abordagens propostas. As soluções de integração foram testadas e validadas com recurso ao simulador Webots e ao modelo do robô DARwIN-OP.In this work, it is intended to apply Reinforcement Learning techniques in tasks involving learning and robot locomotion. Reinforcement Learning is a very useful learning technique with regard to legged robot locomotion, due to its ability to provide direct interaction between the agent and the environment, and the fact of not requiring supervision or complete models, in contrast with other classic approaches. Its aim consists in making decisions about which actions to take so as to maximize a cumulative reward or reinforcement signal, taking into account the fact that the decisions may affect not only the immediate reward, but also the future ones. In this work it will be studied and presented the Reinforcement Learning framework and its application in the tuning of Central Pattern Generators, with the aim of generating optimized robot locomotion. In order to investigate the strengths and abilities of Reinforcement Learning, and to demonstrate in a simple way the learning process of such algorithms, two case studies were implemented based on the state-of-the-art. With regard to the main purpose of the thesis, two different solutions are addressed: a first one based on Natural-Actor Critic methods, and a second, based on the Cross-Entropy Method. This last algorithm was found to be very capable of handling with the integration of the two proposed approaches. The integration solutions were tested and validated resorting to Webots simulation and DARwIN-OP robot model

Humanoid Robots

For many years, the human being has been trying, in all ways, to recreate the complex mechanisms that form the human body. Such task is extremely complicated and the results are not totally satisfactory. However, with increasing technological advances based on theoretical and experimental researches, man gets, in a way, to copy or to imitate some systems of the human body. These researches not only intended to create humanoid robots, great part of them constituting autonomous systems, but also, in some way, to offer a higher knowledge of the systems that form the human body, objectifying possible applications in the technology of rehabilitation of human beings, gathering in a whole studies related not only to Robotics, but also to Biomechanics, Biomimmetics, Cybernetics, among other areas. This book presents a series of researches inspired by this ideal, carried through by various researchers worldwide, looking for to analyze and to discuss diverse subjects related to humanoid robots. The presented contributions explore aspects about robotic hands, learning, language, vision and locomotion