Probabilistic and Distributed Control of a Large-Scale Swarm of Autonomous Agents

We present a novel method for guiding a large-scale swarm of autonomous agents into a desired formation shape in a distributed and scalable manner. Our Probabilistic Swarm Guidance using Inhomogeneous Markov Chains (PSG-IMC) algorithm adopts an Eulerian framework, where the physical space is partitioned into bins and the swarm's density distribution over each bin is controlled. Each agent determines its bin transition probabilities using a time-inhomogeneous Markov chain. These time-varying Markov matrices are constructed by each agent in real-time using the feedback from the current swarm distribution, which is estimated in a distributed manner. The PSG-IMC algorithm minimizes the expected cost of the transitions per time instant, required to achieve and maintain the desired formation shape, even when agents are added to or removed from the swarm. The algorithm scales well with a large number of agents and complex formation shapes, and can also be adapted for area exploration applications. We demonstrate the effectiveness of this proposed swarm guidance algorithm by using results of numerical simulations and hardware experiments with multiple quadrotors.Comment: Submitted to IEEE Transactions on Robotic

Feedback-Based Inhomogeneous Markov Chain Approach To Probabilistic Swarm Guidance

This paper presents a novel and generic distributed swarm guidance algorithm using inhomogeneous Markov chains that guarantees superior performance over existing homogeneous Markov chain based algorithms, when the feedback of the current swarm distribution is available. The probabilistic swarm guidance using inhomogeneous Markov chain (PSG–IMC) algorithm guarantees sharper and faster convergence to the desired formation or unknown target distribution, minimizes the number of transitions for achieving and maintaining the formation even if the swarm is damaged or agents are added/removed from the swarm, and ensures that the agents settle down after the swarm’s objective is achieved. This PSG–IMC algorithm relies on a novel technique for constructing Markov matrices for a given stationary distribution. This technique incorporates the feedback of the current swarm distribution, minimizes the coefficient of ergodicity and the resulting Markov matrix satisfies motion constraints. This approach is validated using Monte Carlo simulations of the PSG–IMC algorithm for pattern formation and goal searching application

Control of Large Swarms via Random Finite Set Theory

Controlling large swarms of robotic agents has many challenges including, but not limited to, computational complexity due to the number of agents, uncertainty in the functionality of each agent in the swarm, and uncertainty in the swarm's configuration. This work generalizes the swarm state using Random Finite Set (RFS) theory and solves the control problem using model predictive control which naturally handles the challenges. This work uses information divergence to define the distance between swarm RFS and a desired distribution. A stochastic optimal control problem is formulated using a modified L2 distance. Simulation results show that swarm densities converge to a target destination, and the RFS control formulation can vary in the number of target destinations

Random Finite Set Theory and Optimal Control of Large Collaborative Swarms

Controlling large swarms of robotic agents has many challenges including, but not limited to, computational complexity due to the number of agents, uncertainty in the functionality of each agent in the swarm, and uncertainty in the swarm's configuration. This work generalizes the swarm state using Random Finite Set (RFS) theory and solves the control problem using Model Predictive Control (MPC) to overcome the aforementioned challenges. Computationally efficient solutions are obtained via the Iterative Linear Quadratic Regulator (ILQR). Information divergence is used to define the distance between the swarm RFS and the desired swarm configuration. Then, a stochastic optimal control problem is formulated using a modified L2^2 distance. Simulation results using MPC and ILQR show that swarm intensities converge to a target destination, and the RFS control formulation can vary in the number of target destinations. ILQR also provides a more computationally efficient solution to the RFS swarm problem when compared to the MPC solution. Lastly, the RFS control solution is applied to a spacecraft relative motion problem showing the viability for this real-world scenario.Comment: arXiv admin note: text overlap with arXiv:1801.0731

Feedback-Based Inhomogeneous Markov Chain Approach To Probabilistic Swarm Guidance

This paper presents a novel and generic distributed swarm guidance algorithm using inhomogeneous Markov chains that guarantees superior performance over existing homogeneous Markov chain based algorithms, when the feedback of the current swarm distribution is available. The probabilistic swarm guidance using inhomogeneous Markov chain (PSG–IMC) algorithm guarantees sharper and faster convergence to the desired formation or unknown target distribution, minimizes the number of transitions for achieving and maintaining the formation even if the swarm is damaged or agents are added/removed from the swarm, and ensures that the agents settle down after the swarm’s objective is achieved. This PSG–IMC algorithm relies on a novel technique for constructing Markov matrices for a given stationary distribution. This technique incorporates the feedback of the current swarm distribution, minimizes the coefficient of ergodicity and the resulting Markov matrix satisfies motion constraints. This approach is validated using Monte Carlo simulations of the PSG–IMC algorithm for pattern formation and goal searching application

Effective task allocation frameworks for large-scale multiple agent systems.

This research aims to develop innovative and transformative decision-making frameworks that enable a large-scale multi-robot system, called robotic swarm, to autonomously address multi-robot task allocation problem: given a set of complicated tasks, requiring cooperation, how to partition themselves into subgroups (or called coalitions) and assign the subgroups to each task while maximising the system performance. The frameworks should be executable based on local information in a decentralised manner, operable for a wide range of the system size (i.e., scalable), predictable in terms of collective behaviours, adaptable to dynamic environments, operable asynchronously, and preferably able to accommodate heterogeneous agents. Firstly, for homogeneous robots, this thesis proposes two frameworks based on biological inspiration and game theories, respectively. The former, called LICA-MC (Markov-Chan-based approach under Local Information Consistency Assumption), is inspired by fish in nature: despite insufficient awareness of the entire group, they are well-coordinated by sensing social distances from neighbours. Analogously, each agent in the framework relies only on local information and requires its local consistency over neighbouring agents to adaptively generate the stochastic policy. This feature offers various advantages such as less inter-agent communication, a shorter timescale for using new information, and the potential to accommodate asynchronous behaviours of agents. We prove that the agents can converge to a desired collective status without resorting to any global information, while maintaining scalability, flexibility, and long-term system efficiency. Numerical experiments show that the framework is robust in a realistic environment where information sharing over agents is partially and temporarily disconnected. Furthermore, we explicitly present the design requirements to have all these advantages, and implementation examples concerning travelling costs minimisation, over-congestion avoidance, and quorum models, respectively. The game-theoretical framework, called GRAPE (GRoup Agent Partitioning and placing Event), regards each robot as a self-interested player attempting to join the most preferred coalition according to its individual preferences regarding the size of each coalition. We prove that selfish agents who have social inhibition can always converge to a Nash stable partition (i.e., a social agreement) within polynomial time under the proposed framework. The framework is executable based on local interactions with neighbour agents under a strongly-connected communication network and even in asynchronous environments. This study analyses an outcome’s minimum-guaranteed suboptimality, and additionally shows that at least 50% is guaranteed if social utilities are non-decreasing functions with respect to the number of co-working agents. Numerical experiments confirm that the framework is scalable, fast adaptable against dynamical environments, and robust even in a realistic situation where some of the agents temporarily halt operation during a mission. The two proposed frameworks are compared in the domain of division of labour. Empirical results show that LICA-MC provides excellent scalability with respect to the number of agents, whereas GRAPE has polynomial complexity but is more efficient in terms of convergence time (especially when accommodating a moderate number of robots) and total travelling costs. It also turns out that GRAPE is sensitive to traffic congestion, meanwhile LICA-MC suffers from slower robot speed. We discuss other implicit advantages of the frameworks such as mission suitability and additionally-builtin decision-making functions. Importantly, it is found that GRAPE has the potential to accommodate heterogeneous agents to some extent, which is not the case for LICA-MC. Accordingly, this study attempts to extend GRAPE to incorporate the heterogeneity of agents. Particularly, we consider the case where each task has its minimum workload requirement to be fulfilled by multiple agents and the agents have different work capacities and costs depending on the tasks. The objective is to find an assignment that minimises the total cost of assigned agents while satisfying the requirements. GRAPE cannot be directly used because of the heterogeneity, so we adopt tabu-learning heuristics where an agent penalises its previously chosen coalition whenever it changes decision: this variant is called T-GRAPE. We prove that, by doing so, a Nash stable partition is always guaranteed to be determined in a decentralised manner. Experi-mental results present the properties of the proposed approach regarding suboptimality and algorithmic complexity. Finally, the thesis addresses a more complex decision-making problem involving team formation, team-to-task assignment, agent-to-working-position selection, fair resource allocation concerning tasks’ minimum requirements for completion, and trajectory optimisation with collision avoidance. We propose an integrated framework that decouples the original problem into three subproblems (i.e., coalition formation, position allocation, and path planning) and deals with them sequentially by three respective modules. The coalition formation module based on T-GRAPE deals with a max-min problem, balancing the work resources of agents in proportion to the task’s requirements. We show that, given reasonable assumptions, the position allocation subproblem can be solved efficiently in terms of computational complexity. For the path planning, we utilise an MPC-SCP (Model Predictive Control and Sequential Convex Programming) approach that enables the agents to produce collision-free trajectories. As a proof of concept, we implement the framework into a cooperative stand-in jamming mission scenario using multiple UAVs. Numerical experiments suggest that the framework could be computationally feasible, fault-tolerant, and near-optimal. Comparison of the proposed frameworks for multi-robot task allocation is discussed in the last chapter regarding the desired features described at first (i.e., decentralisation, scalability, predictability, flexibility, asynchronisation, heterogeneity), along with future work and possible applications in other domains.PhD in Aerospac