Intelligent System Synthesis for Dynamic Locomotion Behavior in Multi-legged Robots

Robot technology has been implemented in many fields of our life, such as entertainment, security, rescue, rehabilitation, social life, the military, and etc. Multi-legged robot always exist in many fields, therefore it is important to be developed. Motion capabilities of the robot will be a main focus to be developed. Current development or conventional model of motion capabilities have several issues in saturation of development. There are some limitation in dynamic factors such as, locomotion generator, flexibility of motion planning, and smoothness of movement. Therefore, in this research, natural based computation are implemented as the basic model. There are three subsystems to be developed and integrated, (1) locomotion behavior model, (2) stability behavior model, and (3) motion planning model. Since individual people has different walking behavior in each walking direction and walking speed, locomotion behavior learning model of omni-directional bio-inspired locomotion which is generating different walking behavior in different walking provision are required to be developed. Step length in sagital and coronal direction, and degree of turning are considered parameters in walking provision. In proposed omni-directional walking model, interconnection structures composed by 16 neurons where 1 leg is represented by 4 joints and 1 joint is represented by 2 motor neurons. In order to acquire walking behavior in certain walking provision, the interconnection structure is optimized by multi-objectives evolutionary algorithm. For acquiring the diversity of references, several optimized interconnection structures are generated in optimization processes in different walking provisions. Learning models are proposed for solving non-linearity of relationship between walking input and walking output representing the synaptic weight of interconnection structure, where one learning model representing one walking parameter. Furthermore, by using optimized model, walking behavior can be generated with unsealed walking provision. Smooth walking transition with low error of desired walking provision was proved based on several numerical experiments in physical computer simulation. In stability behavior model, neuro-based push recovery controller is applied in multi-legged robot in order to keep the stability with minimum energy required. There are three motion patterns in individual people behavior when it gets external perturbation, those are ankle behavior, hip behavior, and step behavior. We propose a new model of Modular Recurrent Neural Network (MRNN) for performing online learning system in each motion behavior. MRNN consists of several recurrent neural networks (RNNs) working alternatively depending on the condition. MRNN performs online learning process of each motion behavior controller independently. The aim of push recovery controller is to manage the motion behavior controller by minimizing the energy required for responding to the external perturbation. This controller selects the appropriate motion behavior and adjusts the gain that represent the influence of the motion behavior to certain push disturbance based on behavior graphs which is generated by adaptive regression spline. We applied the proposed controller to the humanoid robot that has small footprint in open dynamics engine. Experimental result shows the effectiveness of the push controller stabilizing the external perturbation with minimum energy required. Proposed motion planning model presents a natural mechanism of the human brain for generating a dynamic path planning in 3-D rough terrain. The proposed model not only emphasizes the inner state process of the neuron but also the development process of the neurons in the brain. There are two information transmission processes in this proposed model, the forward transmission activity for constructing the neuron connections to find the possible way and the synaptic pruning activity with backward neuron transmission for finding the best pathway from current position to target position and reducing inefficient neuron with its synaptic connections. In order to respond and avoid the unpredictable obstacle, dynamic path planning is also considered in this proposed model. An integrated system for applying the proposed model in the actual experiments is also presented. In order to confirm the effectiveness of the proposed model, we applied the integrated system in the pathway of a four-legged robot on rough terrain in computer simulation. For analyzing and proving the flexibility of proposed model, unpredictable collision is also performed in those experiments. The model can find the best pathway and facilitate the safe movement of the robot. When the robot found an unpredictable collision, the path planner dynamically changed the pathway. The proposed path planning model is capable to be applied in further advance implementation. In order to implement the motion capabilities in real cases, all subsystem should be integrated into one interconnected motion capabilities model. We applied small quadruped robot equipped with IMU, touch sensor, and dual ultrasonic sensor for performing motion planning in real terrain from starting point to goal point. Before implemented, topological map is generated by Kinect camera. In this implementation, all subsystem were analyzed and performed well and the robot able to stop in the goal point. These implementation proved the effectiveness of the system integration, the motion planning model is able to generate safe path planning, the locomotion model is able to generate flexible movement depending on the walking provision from motion planning model, and the stability model can stabilize the robot on rough terrain. Generally, the proposed model can be expected to bring a great contribution to the motion capabilities development and can be used as alternative model for acquiring the dynamism and efficient model in the future instead of conventional model usage. In the future, the proposed model can be applied into any legged robot as navigation, supporter, or rescue robot in unstable environmental condition. In addition, we will realize a cognitive locomotion that generates multiple gaits depending on the 3 aspects, embodiment, locomotion generator, and cognition model. A dynamic neuro-locomotion integrated with internal and external sensory information for correlating with the environmental condition will be designed.ロボット技術は、エンターテイメント、セキュリティ、救助、リハビリ、社会生活、軍事などの様々な生活分野に実現さている。多脚ロポットは常に多くの分野に存在するため開発することが重要である。ロボットの運動能力が開発の主要となっている。現状の開発されている動作能力は,飽和状態にある。いくつかの動的な要因により、歩行生成器、動作計画の柔軟性、および動作の滑らかさ等に制限がある。そこで、本研究では、基本的なモデルとして自然計算に基づく方法論を実装する、また、本研究では、歩行動作モデル、安定動作モデル、や運動計画モデルからなる3つのサブシステムを開発し統合する。人間は歩行方向と速度に応じて歩行動作が異なるため、異なる歩行軸では異なる歩行動作を生成するという全方位生物的な運動の歩行動作学習モデルが開発には要求される。球欠および制御方向のステップ長や旋回の度合いは,歩行軸のパラメータとして考慮される。提案した全方位歩行モデルでは,1肢につき16個のニューロンによって構成される相互接続構造を4つの関節によって表現する。また、1つの関節は,2個のモータニューロンによって表現する。一定の歩行軸での歩行動作を獲得するために,本研究では,多目的進化アルゴリズムによって最適化を行う。提案手法では、参照点の多様性を獲得するために,異なる歩行軸においていくつかの最適な相互接続構造が生成される。相互接続構造のシナプス重みを表現している歩行入力と出力間の非線形な関係を解くための学習モデルを構築する。本手法では,1つの学習モデルが1つの歩行パラメータで表現され、最適化されたモデルを用いることにより,歩行動作は,スケーリングされていない歩行軸を生成することが可能となる,物理演算シミュレーションを用いた実験により,誤差の少ない歩行軸の滑らかな歩行遷移を本実験では示している。安定動作モデルでは、必要最小限のエネルギーで安定性を維持するため多足歩行ロボットにニューロベースプッシュリカバリ制御器を適用した。外力をを受けたとき,人間の行動には足首の動作・股関節の動作・踏み動作の3つの動作パターンが存在する。本研究では,各運動動作におけるオンライン学習システムを実現するために、モジュラーリカレントニューラルネットワーク(MRNN)を用いた新たな学習モデルを提案する。MRNNは状況に応じて選択される複数のリカレントニューラルネットワーク(RNN)によって構成される。MRNNは各運動動作コントローラのオンライン学習プロセスを独立して実行する。プッシュリカバリ制御器の目的は、外乱に応じてエネルギー最小化を行うことによって運動動作制御器を管理することである。この制御器は適切な運動動作を選択し,適応回帰スプラインにより生成された動作グラフに基づき押し動作に対して最も影響を及ぼす運動動作のゲインの調整を行う。提案した制御器をOpen Dynamics Engine(ODE)上で小さな足の長さを持つヒューマノイドロボットに適用し,必要最小限のエネルギーで外力に対して安定させるプッシュリカバリ制御器の有効性を示している。3次元の不整地における動的な経路計画を生成するために,人間の自然な脳機能に基づいた動作計画手法を提案する。本モデルは、ニューロンの内部状態過程だけでなく、脳内のニューロンの発達過程も重視している。本モデルは二つのアルゴリズムに構成される。1つは、通過可能な道を見つけるために構築される接続的なニューロン活動である順方向伝達活動であり,もう1つは、現在位置から最適経路を見つけるために、シナプス結合を用いて非効率的なニューロンを減少させる逆方向にニューロン伝達を行うシナプスプルーニング活動である。また,予測不可能な衝突を回避するために,動的な経路計画も実行される。さらに、実環境において提案されたモデルを実現するための統合システムも提示される。提案モデルの有効性を検証するために,コンピュータシミュレーション上で、不整地環境の4足歩行ロボットに関するシミュレーション環境を実装した。これらの実験では,予測不能な衝突に関する実験も行った。本モデルは、最適経路を見つけ出しロボットの安全な移動を実現できた。さらに、ロボットが予測できない衝突を検出した場合,経路計画アルゴリズムが経路を動的に変更可能であることを示している。これらのことから、提案された経路計画モデルはさらなる先進的な展開が実現可能であると考えられる。実環境における運動能力を実装するためには、すべてのサブシステムを1つの運動能力モデルに統合する必要がある。そこで本研究では、IMU、タッチセンサ、2つの超音波センサを搭載した小型の4足歩行ロポットを用いた実環境において出発地点から目的地点までの運動計画を行った、本実装では、3次元距離計測センサであるKinecを用い3次元空間の位相構造を生成する。また、本実装では、すべてのサブシステムが分析され、ロボットは目的地点で停止することができた。さらに、安全な経路計画を生成することができたことからシステム統合の有効性が確認できた。また、歩行モデルにより歩行軸に応じた柔軟な動きが生成されることで、この安定性モデルは不整地環撹でもロボットの歩行を安定させることができた。これらのことから、本提案モデルは運動能力への多大な貢献が期待され、ダイナミクスを獲得するための代替モデルとして使用することができ,現在よく使用されているモデルに代わる効率的なモデルとなることが考えられる。今後の課題としては,不安定な環境下におけるナビゲーション・支援・レスキューロボットといった任意の肢の数を持つ多足歩行ロボットへの本提案モデルの適用があげられる。さらに,身体性,歩行生成,認知モデルの3つの観点から複数の歩容を生成する認知的歩行を実現することを考えている。環境と相互作用するためのモデルとして、内界センサと外界センサ情報を統合した動的ニューロ歩行を実現する予定である。首都大学東京, 2018-03-25, 修士（工学）首都大学東

Streamlined sim-to-real transfer for deep-reinforcement learning in robotics locomotion

Legged robots possess superior mobility compared to other machines, yet designing controllers for them can be challenging. Classic control methods require engineers to distill their knowledge into controllers, which is time-consuming and limiting when approaching dynamic tasks in unknown environments. Conversely, learning- based methods that gather knowledge from data can potentially unlock the versatility of legged systems. In this thesis, we propose a novel approach called CPG-Actor, which incor- porates feedback into a fully differentiable Central Pattern Generator (CPG) formulation using neural networks and Deep-Reinforcement Learning (RL). This approach achieves approximately twenty times better training performance compared to previous methods and provides insights into the impact of training on the distribution of parameters in both the CPGs and MLP feedback network. Adopting Deep-RL to design controllers comes at the expense of gathering extensive data, typically done in simulation to reduce time. However, controllers trained with data collected in simulation often lose performance when deployed in the real world, referred to as the sim-to-real gap. To address this, we propose a new method called Extended Random Force Injection (ERFI), which randomizes only two parameters to allow for sim-to-real transfer of locomotion controllers. ERFI demonstrated high robustness when varying masses of the base, or attaching a manipulator arm to the robot during testing, and achieved competitive performance comparable to standard randomization techniques. Furthermore, we propose a new method called Roll-Drop to enhance the robustness of Deep-RL policies to observation noise. Roll-Drop introduces dropout during rollout, achieving an 80% success rate when tested with up to 25% noise injected in the observations. Finally, we adopted model-free controllers to enable omni-directional bipedal lo- comotion on point feet with a quadruped robot without any hardware modification or external support. Despite the limitations posed by the quadruped’s hardware, the study considers this a perfect benchmark task to assess the shortcomings of sim- to-real techniques and unlock future avenues for the legged robotics community. Overall, this thesis demonstrates the potential of learning-based methods to design dynamic and robust controllers for legged robots while limiting the effort needed for sim-to-real transfer

분산된 로터로 구동되는 비행 스켈레톤 시스템의 디자인 상태추정 및 제어

학위논문(박사)--서울대학교 대학원 :공과대학 기계항공공학부,2020. 2. 이동준.In this thesis, we present key theoretical components for realizing flying aerial skeleton system called LASDRA (large-size aerial skeleton with distributed rotor actuation). Aerial skeletons are articulated aerial robots actuated by distributed rotors including both ground connected type and flying type. These systems have recently attracted interest and are being actively researched in several research groups, with the expectation of applying those for aerial manipulation in distant/narrow places, or for the performance with entertaining purpose such as drone shows. Among the aerial skeleton systems, LASDRA system, proposed by our group has some significant advantages over the other skeleton systems that it is capable of free SE(3) motion by omni-directional wrench generation of each link, and also the system can be operated with wide range of configuration because of the 3DOF (degrees of freedom) inter-link rotation enabled by cable connection among the link modules. To realize this LASDRA system, following three components are crucial: 1) a link module that can produce omni-directional force and torque and enough feasible wrench space; 2) pose and posture estimation algorithm for an articulated system with high degrees of freedom; and 3) a motion generation framework that can provide seemingly natural motion while being able to generate desired motion (e.g., linear and angular velocity) for the entire body. The main contributions of this thesis is theoretically developing these three components, and verifying these through outdoor flight experiment with a real LASDRA system. First of all, a link module for the LASDRA system is designed with proposed constrained optimization problem, maximizing the guaranteed feasible force and torque for any direction while also incorporating some constraints (e.g., avoiding inter-rotor air-flow interference) to directly obtain feasible solution. Also, an issue of ESC-induced (electronic speed control) singularity is first introduced in the literature which is inevitably caused by bi-directional thrust generation with sensorless actuators, and handled with a novel control allocation called selective mapping. Then for the state estimation of the entire LASDRA system, constrained Kalman filter based estimation algorithm is proposed that can provide estimation result satisfying kinematic constraint of the system, also along with a semi-distributed version of the algorithm to endow with system scalability. Lastly, CPG-based motion generation framework is presented that can generate natural biomimetic motion, and by exploiting the inverse CPG model obtained with machine learning method, it becomes possible to generate certain desired motion while still making CPG generated natural motion.본 논문에서는 비행 스켈레톤 시스템 LASDRA (large-size aerial skeleton with distributed rotor actuation) 의 구현을 위해 요구되는 핵심 기법들을 제안하며, 이를 실제 LASDRA 시스템의 실외 비행을 통해 검증한다. 제안된 기법은 1) 전방향으로 힘과 토크를 낼 수 있고 충분한 가용 렌치공간을 가진 링크 모듈, 2) 높은 자유도의 다관절구조 시스템을 위한 위치 및 자세 추정 알고리즘, 3) 자연스러운 움직임을 내는 동시에 전체 시스템이 속도, 각속도 등 원하는 움직임을 내도록 할 수 있는 모션 생성 프레임워크로 구성된다. 본 논문에서는 우선 링크 모듈의 디자인을 위해 전방향으로 보장되는 힘과 토크의 크기를 최대화하는 구속 최적화를 사용하고, 실제 적용가능한 해를 얻기 위해 몇가지 구속조건(로터 간 공기 흐름 간섭의 회피 등)을 고려한다. 또한 센서가 없는 액츄에이터로 양방향 추력을 내는 것에서 야기되는 ESC 유발 특이점 (ESC-induced singularity) 이라는 문제를 처음으로 소개하고, 이를 해결하기 위해 선택적 맵핑 (selective mapping) 이라는 기법을 제시한다. 전체 LASDRA 시스템의 상태추정을 위해 시스템의 기구학적 구속조건을 만족하는 결과를 얻을 수 있도록 구속 칼만 필터 기반의 상태추정 기법을 제시하고, 시스템 확장성을 고려하여 반 분산 (semi-distributed) 개념의 알고리즘을 함께 제시한다. 마지막으로 본 논문에서는 자연스러운 움직임의 생성을 위하여 CPG 기반의 모션 생성 프레임워크를 제안하며, 기계 학습 방법을 통해 CPG 역연산 모델을 얻음으로써 전체 시스템이 원하는 움직임을 낼 수 있도록 한다.1 Introduction 1 1.1 Motivation and Background 1 1.2 Research Problems and Approach 3 1.3 Preview of Contributions 5 2 Omni-Directional Aerial Robot 7 2.1 Introduction 7 2.2 Mechanical Design 12 2.2.1 Design Description 12 2.2.2 Wrench-Maximizing Design Optimization 13 2.3 System Modeling and Control Design 20 2.3.1 System Modeling 20 2.3.2 Pose Trajectory Tracking Control 22 2.3.3 Hybrid Pose/Wrench Control 22 2.3.4 PSPM-Based Teleoperation 24 2.4 Control Allocation with Selective Mapping 27 2.4.1 Infinity-Norm Minimization 27 2.4.2 ESC-Induced Singularity and Selective Mapping 29 2.5 Experiment 38 2.5.1 System Setup 38 2.5.2 Experiment Results 41 2.6 Conclusion 49 3 Pose and Posture Estimation of an Aerial Skeleton System 51 3.1 Introduction 51 3.2 Preliminary 53 3.3 Pose and Posture Estimation 55 3.3.1 Estimation Algorithm via SCKF 55 3.3.2 Semi-Distributed Version of Algorithm 59 3.4 Simulation 62 3.5 Experiment 65 3.5.1 System Setup 65 3.5.2 Experiment of SCKF-Based Estimation Algorithm 66 3.6 Conclusion 69 4 CPG-Based Motion Generation 71 4.1 Introduction 71 4.2 Description of Entire Framework 75 4.2.1 LASDRA System 75 4.2.2 Snake-Like Robot & Pivotboard 77 4.3 CPG Model 79 4.3.1 LASDRA System 79 4.3.2 Snake-Like Robot 80 4.3.3 Pivotboard 83 4.4 Target Pose Calculation with Expected Physics 84 4.5 Inverse Model Learning 86 4.5.1 LASDRA System 86 4.5.2 Snake-Like Robot 89 4.5.3 Pivotboard 90 4.6 CPG Parameter Adaptation 93 4.7 Simulation 94 4.7.1 LASDRA System 94 4.7.2 Snake-Like Robot & Pivotboard 97 4.8 Conclusion 101 5 Outdoor Flight Experiment of the F-LASDRA System 103 5.1 System Setup 103 5.2 Experiment Results 104 6 Conclusion 111 6.1 Summary 111 6.2 Future Works 112Docto

Trends in the control of hexapod robots: a survey

The static stability of hexapods motivates their design for tasks in which stable locomotion is required, such as navigation across complex environments. This task is of high interest due to the possibility of replacing human beings in exploration, surveillance and rescue missions. For this application, the control system must adapt the actuation of the limbs according to their surroundings to ensure that the hexapod does not tumble during locomotion. The most traditional approach considers their limbs as robotic manipulators and relies on mechanical models to actuate them. However, the increasing interest in model-free models for the control of these systems has led to the design of novel solutions. Through a systematic literature review, this paper intends to overview the trends in this field of research and determine in which stage the design of autonomous and adaptable controllers for hexapods is.The first author received funding through a doctoral scholarship from the Portuguese Foundation for Science and Technology (FCT) (Grant No. SFRH/BD/145818/2019), with funds from the Portuguese Ministry of Science, Technology and Higher Education and the European Social Fund through the Programa Operacional Regional Norte. This work has been supported by the FCT national funds, under the national support to R&D units grant, through the reference project UIDB/04436/2020 and UIDP/04436/2020

Virtual reality-based assessment and rehabilitation of functional mobility

The advent of virtual reality (VR) as a tool for real-world training dates back to the mid-twentieth century and the early years of driving and flight simulators. These simulation environments, while far below the quality of today’s visual displays, proved to be advantageous to the learner due to the safe training environments the simulations provided. More recently, these training environments have proven beneficial in the transfer of user-learned skills from the simulated environment to the real world [5, 31, 48, 51, 57]. Of course the VR technology of today has come a long way. Contemporary displays boast high-resolution, wide-angle fields of view and increased portability. This has led to the evolution of new VR research and training applications in many different arenas, several of which are covered in other chapters of this book. This is true of clinical assessment and rehabilitation as well, as the field has recognized the potential advantages of incorporating VR technologies into patient training for almost 20 years [7, 10, 18, 45, 78]

Bio-Inspired Robotics

Modern robotic technologies have enabled robots to operate in a variety of unstructured and dynamically-changing environments, in addition to traditional structured environments. Robots have, thus, become an important element in our everyday lives. One key approach to develop such intelligent and autonomous robots is to draw inspiration from biological systems. Biological structure, mechanisms, and underlying principles have the potential to provide new ideas to support the improvement of conventional robotic designs and control. Such biological principles usually originate from animal or even plant models, for robots, which can sense, think, walk, swim, crawl, jump or even fly. Thus, it is believed that these bio-inspired methods are becoming increasingly important in the face of complex applications. Bio-inspired robotics is leading to the study of innovative structures and computing with sensory–motor coordination and learning to achieve intelligence, flexibility, stability, and adaptation for emergent robotic applications, such as manipulation, learning, and control. This Special Issue invites original papers of innovative ideas and concepts, new discoveries and improvements, and novel applications and business models relevant to the selected topics of ``Bio-Inspired Robotics''. Bio-Inspired Robotics is a broad topic and an ongoing expanding field. This Special Issue collates 30 papers that address some of the important challenges and opportunities in this broad and expanding field

Actuation-Aware Simplified Dynamic Models for Robotic Legged Locomotion

In recent years, we witnessed an ever increasing number of successful hardware implementations of motion planners for legged robots. If one common property is to be identified among these real-world applications, that is the ability of online planning. Online planning is forgiving, in the sense that it allows to relentlessly compensate for external disturbances of whatever form they might be, ranging from unmodeled dynamics to external pushes or unexpected obstacles and, at the same time, follow user commands. Initially replanning was restricted only to heuristic-based planners that exploit the low computational effort of simplified dynamic models. Such models deliberately only capture the main dynamics of the system, thus leaving to the controllers the issue of anchoring the desired trajectory to the whole body model of the robot. In recent years, however, we have seen a number of new approaches attempting to increase the accuracy of the dynamic formulation without trading-off the computational efficiency of simplified models. In this dissertation, as an example of successful hardware implementation of heuristics and simplified model-based locomotion, I describe the framework that I developed for the generation of an omni-directional bounding gait for the HyQ quadruped robot. By analyzing the stable limit cycles for the sagittal dynamics and the Center of Pressure (CoP) for the lateral stabilization, the described locomotion framework is able to achieve a stable bounding while adapting to terrains of mild roughness and to sudden changes of the user desired linear and angular velocities. The next topic reported and second contribution of this dissertation is my effort to formulate more descriptive simplified dynamic models, without trading off their computational efficiency, in order to extend the navigation capabilities of legged robots to complex geometry environments. With this in mind, I investigated the possibility of incorporating feasibility constraints in these template models and, in particular, I focused on the joint torques limits which are usually neglected at the planning stage. In this direction, the third contribution discussed in this thesis is the formulation of the so called actuation wrench polytope (AWP), defined as the set of feasible wrenches that an articulated robot can perform given its actuation limits. Interesected with the contact wrench cone (CWC), this yields a new 6D polytope that we name feasible wrench polytope (FWP), defined as the set of all wrenches that a legged robot can realize given its actuation capabilities and the friction constraints. Results are reported where, thanks to efficient computational geometry algorithms and to appropriate approximations, the FWP is employed for a one-step receding horizon optimization of center of mass trajectory and phase durations given a predefined step sequence on rough terrains. For the sake of reachable workspace augmentation, I then decided to trade off the generality of the FWP formulation for a suboptimal scenario in which a quasi-static motion is assumed. This led to the definition of the, so called, local/instantaneous actuation region and of the global actuation/feasible region. They both can be seen as different variants of 2D linear subspaces orthogonal to gravity where the robot is guaranteed to place its own center of mass while being able to carry its own body weight given its actuation capabilities. These areas can be intersected with the well known frictional support region, resulting in a 2D linear feasible region, thus providing an intuitive tool that enables the concurrent online optimization of actuation consistent CoM trajectories and target foothold locations on rough terrains