Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

Technical Report on: Tripedal Dynamic Gaits for a Quadruped Robot

A vast number of applications for legged robots entail tasks in complex, dynamic environments. But these environments put legged robots at high risk for limb damage. This paper presents an empirical study of fault tolerant dynamic gaits designed for a quadrupedal robot suffering from a single, known ``missing'' limb. Preliminary data suggests that the featured gait controller successfully anchors a previously developed planar monopedal hopping template in the three-legged spatial machine. This compositional approach offers a useful and generalizable guide to the development of a wider range of tripedal recovery gaits for damaged quadrupedal machines.Comment: Updated *increased font size on figures 2-6 *added a legend, replaced text with colors in figure 5a and 6a *made variables representing vectors boldface in equations 8-10 *expanded on calculations in equations 8-10 by adding additional lines *added a missing "2" to equation 8 (typo) *added mass of the robot to tables II and III *increased the width of figures 1 and

Robot Berkaki Empat Pendeteksi Cahaya dan Penghindar Rintangan

Robotika merupakan bidang ilmu pengetahuan yang mengalami perkembangan dengan cepat, salah satunya adalah robot berkaki empat. Pembahasan mengenai robot berkaki meliputi banyak hal seperti fungsi dan kegunaannya, metode kontrol yang digunakan dan lain-lain. Permasalahan yang muncul pada robot berkaki empat ini adalah cara mendeteksi adanya cahaya dan cara menghindari sebuah rintangan di depan robot. Tujuan utama dari tugas akhir ini adalah menggunakan metode kendali yang baik agar robot bisa mendeteksi adanya cahaya dan dapat menghindari rintangan di depan robot tanpa menabrak rintangan yang menghalanginya. Robot berkaki ini menggunakan sensor LDR yaitu untuk mendeteksi adanya cahaya dan sensor ultrasonik (PINGTM) yaitu untuk mendeteksi ada tidaknya rintangan sehingga robot berkaki mampu berjalan tanpa menabrak rintangan yang menghalanginya. Sedangkan untuk penggeraknya (kaki) digunakan motor servo. Pada tugas akhir ini, permasalahan pada robot berkaki diselesaikan dengan menerapkan metode kendali Proporsional Derivatif (PD). Kata kunci : Robot berkaki, Kendali Proporsional Derivatif, Sensor LDR, Sensor jara

Compulsory Flow Q-Learning: an RL algorithm for robot navigation based on partial-policy and macro-states

Reinforcement Learning is carried out on-line, through trial-and-error interactions of the agent with the environment, which can be very time consuming when considering robots. In this paper we contribute a new learning algorithm, CFQ-Learning, which uses macro-states, a low-resolution discretisation of the state space, and a partial-policy to get around obstacles, both of them based on the complexity of the environment structure. The use of macro-states avoids convergence of algorithms, but can accelerate the learning process. In the other hand, partial-policies can guarantee that an agent fulfils its task, even through macro-state. Experiments show that the CFQ-Learning performs a good balance between policy quality and learning rate.Coordenacao de Aperfeicoamento de Pessoal de Nivel Superior (CAPES)GRICESFAPESPCNP

Robust and reusable self-organized locomotion of legged robots under adaptive physical and neural communications

IntroductionAnimals such as cattle can achieve versatile and elegant behaviors through automatic sensorimotor coordination. Their self-organized movements convey an impression of adaptability, robustness, and motor memory. However, the adaptive mechanisms underlying such natural abilities of these animals have not been completely realized in artificial legged systems.MethodsHence, we propose adaptive neural control that can mimic these abilities through adaptive physical and neural communications. The control algorithm consists of distributed local central pattern generator (CPG)-based neural circuits for generating basic leg movements, an adaptive sensory feedback mechanism for generating self-organized phase relationships among the local CPG circuits, and an adaptive neural coupling mechanism for transferring and storing the formed phase relationships (a gait pattern) into the neural structure. The adaptive neural control was evaluated in experiments using a quadruped robot.ResultsThe adaptive neural control enabled the robot to 1) rapidly and automatically form its gait (i.e., self-organized locomotion) within a few seconds, 2) memorize the gait for later recovery, and 3) robustly walk, even when a sensory feedback malfunction occurs. It also enabled maneuverability, with the robot being able to change its walking speed and direction. Moreover, implementing adaptive physical and neural communications provided an opportunity for understanding the mechanism of motor memory formation.DiscussionOverall, this study demonstrates that the integration of the two forms of communications through adaptive neural control is a powerful way to achieve robust and reusable self-organized locomotion in legged robots

ロボティクスのための深層強化学習による運動シナジー発現と冗長性定量化に関する研究

Tohoku University林部充宏課

Simplifying robotic locomotion by escaping traps via an active tail

Legged systems offer the ability to negotiate and climb heterogeneous terrains, more so than their wheeled counterparts \cite{freedberg_2012}. However, in certain complex environments, these systems are susceptible to failure conditions. These scenarios are caused by the interplay between the locomotor's kinematic state and the local terrain configuration, thus making them challenging to predict and overcome. These failures can cause catastrophic damage to the system and thus, methods to avoid such scenarios have been developed. These strategies typically take the form of environmental sensing or passive mechanical elements that adapt to the terrain. Such methods come at an increased control and mechanical design complexity for the system, often still being susceptible to imperceptible hazards. In this study, we investigated whether a tail could serve to offload this complexity by acting as a mechanism to generate new terradynamic interactions and mitigate failure via substrate contact. To do so, we developed a quadrupedal C-leg robophysical model (length and width = 27 cm, limb radius = 8 cm) capable of walking over rough terrain with an attachable actuated tail (length = 17 cm). We investigated three distinct tail strategies: static pose, periodic tapping, and load-triggered (power) tapping, while varying the angle of the tail relative to the body. We challenged the system to traverse a terrain (length = 160 cm, width = 80 cm) of randomized blocks (length and width = 10 cm, height = 0 to 12 cm) whose dimensions were scaled to the robot. Over this terrain, the robot exhibited trapping failures independent of gait pattern. Using the tail, the robot could free itself from trapping with a probability of 0 to 0.5, with the load-driven behaviors having comparable performance to low frequency periodic tapping across all tested tail angles. Along with increasing this likelihood of freeing, the robot displayed a longer survival distance over the rough terrain with these tail behaviors. In summary, we present the beginning of a framework that leverages mechanics via tail-ground interactions to offload limb control and design complexity to mitigate failure and improve legged system performance in heterogeneous environments.M.S

Energetics and Passive Dynamics of Quadruped Robot Planar Running Gaits

Quadruped robots find application in military for load carrying over uneven terrain, humanitarian de-mining, and search and rescue missions. The energy required for quadruped robot locomotion needs to be supplied from on-board energy source which can be either electrical batteries or fuels such as gasolene/diesel. The range and duration of missions very much depend on the amount of energy carried, which is highly limited. Hence, energy efficiency is of paramount importance in building quadruped robots. Study of energy efficiency in quadruped robots not only helps in efficient design of quadruped robots, but also helps understand the biomechanics of quadrupedal animals. This thesis focuses on the energy efficiency of planar running gaits and presents: (a) derivation of cost of transport expressions for trot and bounding gaits, (b) advantages of articulated torso over rigid torso for quadruped robot, (c) symmetry based control laws for passive dynamic bounding and design for inherent stability, and (d) effect of asymmetry in zero-energy bounding gaits