Optimal control approaches for consensus and path planning in multi-agent systems

Optimal control is one of the most powerful, important and advantageous topics in control engineering. The two challenges in every optimal control problem are defining the proper cost function and obtaining the best method to minimize it. In this study, innovative optimal control approaches are developed to solve the two problems of consensus and path planning in multi-agent systems (MASs). The consensus problem for general Linear-Time Invariant systems is solved by implementing an inverse optimal control approach which enables us to start by deriving a control law based on the stability and optimality condition and then according to the derived control define the cost function. We will see that this method in which the cost function is not specified a priori as the conventional optimal control design has the benefit that the resulting control law is guaranteed to be both stabilizing and optimal. Three new theorems in related linear algebra are developed to enable us to use the algorithm for all the general LTI systems. The designed optimal control is distributed and only needs local neighbor-to-neighbor information based on the communication topology to make the agents achieve consensus and track a desired trajectory. Path planning problem is solved for a group are Unmanned Aerial Vehicles (UAVs) that are assigned to track the fronts of a fires in a process of wildfire management. We use Partially Observable Markov Decision Process (POMDP) in order to minimize the cost function that is defined according to the tracking error. Here the challenge is designing the algorithm such that (1) the UAVs are able to make decisions autonomously on which fire front to track and (2) they are able to track the fire fronts which evolve over time in random directions. We will see that by defining proper models, the designed algorithms provides real-time calculation of control variables which enables the UAVs to track the fronts and find their way autonomously. Furthermore, by implementing Nominal Belief-state Optimization (NBO) method, the dynamic constraints of the UAVs is considered and challenges such as collision avoidance is addressed completely in the context of POMDP

UltraSwarm: A Further Step Towards a Flock of Miniature Helicopters

We describe further progress towards the development of a MAV (micro aerial vehicle) designed as an enabling tool to investigate aerial flocking. Our research focuses on the use of low cost off the shelf vehicles and sensors to enable fast prototyping and to reduce development costs. Details on the design of the embedded electronics and the modification of the chosen toy helicopter are presented, and the technique used for state estimation is described. The fusion of inertial data through an unscented Kalman filter is used to estimate the helicopter’s state, and this forms the main input to the control system. Since no detailed dynamic model of the helicopter in use is available, a method is proposed for automated system identification, and for subsequent controller design based on artificial evolution. Preliminary results obtained with a dynamic simulator of a helicopter are reported, along with some encouraging results for tackling the problem of flocking

Distributed Control for Collective Behaviour in Micro-unmanned Aerial Vehicles

Full version unavailable due to 3rd party copyright restrictions.The work presented herein focuses on the design of distributed autonomous controllers for collective behaviour of Micro-unmanned Aerial Vehicles (MAVs). Two alternative approaches to this topic are introduced: one based upon the Evolutionary Robotics (ER) paradigm, the other one upon flocking principles. Three computer simulators have been developed in order to carry out the required experiments, all of them having their focus on the modelling of fixed-wing aircraft flight dynamics. The employment of fixed-wing aircraft rather than the omni-directional robots typically employed in collective robotics significantly increases the complexity of the challenges that an autonomous controller has to face. This is mostly due to the strict motion constraints associated with fixed-wing platforms, that require a high degree of accuracy by the controller. Concerning the ER approach, the experimental setups elaborated have resulted in controllers that have been evolved in simulation with the following capabilities: (1) navigation across unknown environments, (2) obstacle avoidance, (3) tracking of a moving target, and (4) execution of cooperative and coordinated behaviours based on implicit communication strategies. The design methodology based upon flocking principles has involved tests on computer simulations and subsequent experimentation on real-world robotic platforms. A customised implementation of Reynolds’ flocking algorithm has been developed and successfully validated through flight tests performed with the swinglet MAV. It has been notably demonstrated how the Evolutionary Robotics approach could be successfully extended to the domain of fixed-wing aerial robotics, which has never received a great deal of attention in the past. The investigations performed have also shown that complex and real physics-based computer simulators are not a compulsory requirement when approaching the domain of aerial robotics, as long as proper autopilot systems (taking care of the ”reality gap” issue) are used on the real robots.EOARD (European Office of Aerospace Research & Development), euCognitio

UAV Formation Flight Utilizing a Low Cost, Open Source Configuration

The control of multiple unmanned aerial vehicles (UAVs) in a swarm or cooperative team scenario has been a topic of great interest for well over a decade, growing steadily with the advancements in UAV technologies. In the academic community, a majority of the studies conducted rely on simulation to test developed control strategies, with only a few institutions known to have nurtured the infrastructure required to propel multiple UAV control studies beyond simulation and into experimental testing. With the Cal Poly UAV FLOC Project, such an infrastructure was created, paving the way for future experimentation with multiple UAV control systems. The control system architecture presented was built on concepts developed in previous work by Cal Poly faculty and graduate students. An outer-loop formation flight controller based on a virtual waypoint implementation of potential function guidance was developed for use on an embedded microcontroller. A commercially-available autopilot system, designed for fully autonomous waypoint navigation utilizing low cost hardware and open source software, was modified to include the formation flight controller and an inter-UAV communication network. A hardware-in-the-loop (HIL) simulation was set up for multiple UAV testing and was utilized to verify the functionality of the modified autopilot system. HIL simulation results demonstrated leader-follower formation convergence to 15 meters as well as formation flight with three UAVs. Several sets of flight tests were conducted, demonstrating a successful leader-follower formation, but with relative distance convergence only reaching a steady state value of approximately 35 +/- 5 meters away from the leader

Multi-Agent Based Simulation of an Unmanned Aerial Vehicles System

The rapid growth of using Unmanned Aerial Vehicles (UAV) for civilian and military applications has promoted the development of research in many areas. Most of the unmanned aerial vehicles in use are manually controlled. Often, UAVs require highly trained pilot operators. Hence, the main challenge faced by researchers has been to make UAVs autonomous or semiautonomous. The goal of this research project is to develop and implement a simulation for a user-defined environment allowing UAVs to maneuver in free environments and obstacle-laden environments using Boid's algorithm of flocking with obstacle avoidance. The users are permitted to analyze the maneuvering area and coverage efficiency of the UAVs and to dynamically change environments. This project makes use of Boid's flocking algorithm to generate different kinds of movements for the flying agents, enabling the user to analyze the effectiveness of patrolling in that particular scenario. The number of UAVs and the type of environment are set by the user. The set number of UAVs moves as a flock or swarm inside the set environment by using Boid's rules of flocking: cohesion, alignment, and separation. The coverage efficiency of the UAVs in that particular environment is reported based on the ratio between the area covered and the time when the search time reaches a threshold. The advantages and feasibilities of the approach are discussed with the simulation results

Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

Motion Planning of UAV Swarm: Recent Challenges and Approaches

The unmanned aerial vehicle (UAV) swarm is gaining massive interest for researchers as it has huge significance over a single UAV. Many studies focus only on a few challenges of this complex multidisciplinary group. Most of them have certain limitations. This paper aims to recognize and arrange relevant research for evaluating motion planning techniques and models for a swarm from the viewpoint of control, path planning, architecture, communication, monitoring and tracking, and safety issues. Then, a state-of-the-art understanding of the UAV swarm and an overview of swarm intelligence (SI) are provided in this research. Multiple challenges are considered, and some approaches are presented. Findings show that swarm intelligence is leading in this era and is the most significant approach for UAV swarm that offers distinct contributions in different environments. This integration of studies will serve as a basis for knowledge concerning swarm, create guidelines for motion planning issues, and strengthens support for existing methods. Moreover, this paper possesses the capacity to engender new strategies that can serve as the grounds for future work

Bearing-only formation control with auxiliary distance measurements, leaders, and collision avoidance

We address the controller synthesis problem for distributed formation control. Our solution requires only relative bearing measurements (as opposed to full translations), and is based on the exact gradient of a Lyapunov function with only global minimizers (independently from the formation topology). These properties allow a simple proof of global asymptotic convergence, and extensions for including distance measurements, leaders and collision avoidance. We validate our approach through simulations and comparison with other stateof-the-art algorithms.ARL grant W911NF-08-2-0004, ARO grant W911NF-13-1-0350, ONR grants N00014-07-1-0829, N00014-14-1-0510, N00014-15-1-2115, NSF grant IIS-1426840, CNS-1521617 and United Technologies

Planning And Control Of Swarm Motion As Continua

In this thesis, new algorithms for formation control of multi agent systems (MAS) based on continuum mechanics principles will be investigated. For this purpose agents of the MAS are treated as particles in a continuum, evolving in an n-D space, whose desired configuration is required to satisfy an admissible deformation function. Considered is a specific class of mappings that is called homogenous where the Jacobian of the mapping is only a function of time and is not spatially varying. The primary objectives of this thesis are to develop the necessary theory and its validation via simulation on a mobile-agent based swarm test bed that includes two primary tasks: 1) homogenous transformation of MAS and 2) deployment of a random distribution of agents on to a desired configuration. Developed will be a framework based on homogenous transformations for the evolution of a MAS in an n-D space (n=1, 2, and 3), under two scenarios: 1) no inter-agent communication (predefined motion plan); and 2) local inter-agent communication. Additionally, homogenous transformations based on communication protocols will be used to deploy an arbitrary distribution of a MAS on to a desired curve. Homogenous transformation with no communication: A homogenous transformation of a MAS, evolving in an space, under zero inter agent communication is first considered. Here the homogenous mapping, is characterized by an n x n Jacobian matrix ( ) and an n x 1 rigid body displacement vector ( ), that are based on positions of n+1 agents of the MAS, called leader agents. The designed Jacobian ( ) and rigid body displacement vector ( ) are passed onto rest of the agents of the MAS, called followers, who will then use that information to update their positions under a pre- iv defined motion plan. Consequently, the motion of MAS will evolve as a homogenous transformation of the initial configuration without explicit communication among agents. Homogenous Transformation under Local Communication: We develop a framework for homogenous transformation of MAS, evolving in , under a local inter agent communication topology. Here we assume that some agents are the leaders, that are transformed homogenously in an n-D space. In addition, every follower agent of the MAS communicates with some local agents to update its position, in order to grasp the homogenous mapping that is prescribed by the leader agents. We show that some distance ratios that are assigned based on initial formation, if preserved, lead to asymptotic convergence of the initial formation to a final formation under a homogenous mapping. Deployment of a Random Distribution on a Desired Manifold: Deployment of agents of a MAS, moving in a plane, on to a desired curve, is a task that is considered as an application of the proposed approach. In particular, a 2-D MAS evolution problem is considered as two 1-D MAS evolution problems, where x or y coordinates of the position of all agents are modeled as points confined to move on a straight line. Then, for every coordinate of MAS evolution, bulk motion is controlled by two agents considered leaders that move independently, with rest of the follower agents motions evolving through each follower agent communicating with two adjacent agents