COOPERATIVE LEARNING FOR THE CONSENSUS OF MULTI-AGENT SYSTEMS

Due to a lot of attention for the multi-agent system in recent years, the consensus algorithm gained immense popularity for building fault-tolerant systems in system and control theory. Generally, the consensus algorithm drives the swarm of agents to work as a coherent group that can reach an agreement regarding a certain quantity of interest, which depends on the state of all agents themselves. The most common consensus algorithm is the average consensus, the final consensus value of which is equal to the average of the initial values. If we want the agents to find the best area of the particular resources, the average consensus will be failure. Thus the algorithm is restricted due to its incapacity to solve some optimization problems. In this dissertation, we want the agents to become more intelligent so that they can handle different optimization problems. Based on this idea, we first design a new consensus algorithm which modifies the general bat algorithm. Since bat algorithm is a swarm intelligence method and is proven to be suitable for solving the optimization problems, this modification is pretty straightforward. The optimization problem suggests the convergence direction. Also, in order to accelerate the convergence speed, we incorporate a term related to flux function, which serves as an energy/mass exchange rate in compartmental modeling or a heat transfer rate in thermodynamics. This term is inspired by the speed-up and speed-down strategy from biological swarms. We prove the stability of the proposed consensus algorithm for both linear and nonlinear flux functions in detail by the matrix paracontraction tool and the Lyapunov-based method, respectively. Another direction we are trying is to use the deep reinforcement learning to train the agent to reach the consensus state. Let the agent learn the input command by this method, they can become more intelligent without human intervention. By this method, we totally ignore the complex mathematical model in designing the protocol for the general consensus problem. The deep deterministic policy gradient algorithm is used to plan the command of the agent in the continuous domain. The moving robots systems are considered to be used to verify the effectiveness of the algorithm. Adviser: Qing Hu

Delay-robust distributed secondary frequency control for next-generation power systems: Stability analysis and controller synthesis

Power systems worldwide are undergoing major transformation to enable a low-carbon future. These developments require new procedures for advanced control to ensure a stable and efficient system operation. Consensus-based distributed secondary frequency control schemes have the potential to ensure real-time frequency restoration and economic dispatch simultaneously in future power systems with significant contribution of renewable energy sources. However, owing to their distributed nature, these control schemes critically depend on communication between different controlled units. Thus, robustness against communication uncertainty is crucial for their reliable operation. In this work, control design and stability analysis of delay-robust secondary frequency control in next-generation power systems are studied. The main contributions of the present thesis can be summarised as follows: (i) A design procedure for a consensus-based secondary frequency controller in microgrids is proposed that ensures robustness with respect to heterogeneous fast-varying communication delays and simultaneously provides the option to trade off the L2-gain performance against the number of required communication links; (ii) The conditions for robust stability of a consensus-based frequency control scheme applied to a power system model with second-order turbine-governor dynamics in the presence of heterogeneous time-varying communication delays and dynamic communication topology are derived; (iii) The performance of the proposed consensus-based secondary frequency controller is analysed in a detailed model capturing the dynamic behaviour of a real system. The results provide insights to the robustness of the closed-loop system with respect to unmodelled (voltage and higher-order generator) dynamics as well as communication delays

Formation Control for a Fleet of Autonomous Ground Vehicles: A Survey

Autonomous/unmanned driving is the major state-of-the-art step that has a potential to fundamentally transform the mobility of individuals and goods. At present, most of the developments target standalone autonomous vehicles, which can sense the surroundings and control the vehicle based on this perception, with limited or no driver intervention. This paper focuses on the next step in autonomous vehicle research, which is the collaboration between autonomous vehicles, mainly vehicle formation control or vehicle platooning. To gain a deeper understanding in this area, a large number of the existing published papers have been reviewed systemically. In other words, many distributed and decentralized approaches of vehicle formation control are studied and their implementations are discussed. Finally, both technical and implementation challenges for formation control are summarized

Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

A comprehensive survey on cultural algorithms

Peer reviewedPostprin

Distributed Secondary Frequency Control Design for Microgrids: Trading Off L2-Gain Performance and Communication Efforts under Time-Varying Delays

Consensus algorithms are promising control schemes for secondary control tasks in microgrids. Since consensus algorithms are distributed protocols, communication efforts and time delays are significant factors for the control design and performance. Moreover, both the electrical and the communication layer in a microgrid are continuously exposed to exogenous disturbances. Motivated by this, we derive a synthesis for a consensus-based secondary frequency controller that guarantees robustness with respect to time-varying delays and in addition provides the option to trade off L2 disturbance attenuation against the number of required communication links. The efficacy of the proposed approach is illustrated via simulations based on the CIGRE benchmark medium voltage distribution network