Online Flocking Control of UAVs with Mean-Field Approximation

We present a novel approach to the formation controlling of aerial robot swarms that demonstrates the flocking behavior. The proposed method stems from the Unmanned Aerial Vehicle (UAV) dynamics; thus, it prevents any unattainable control inputs from being produced and subsequently leads to feasible trajectories. By modeling the inter-agent relationships using a pairwise energy function, we show that interacting robot swarms constitute a Markov Random Field. Our algorithm builds on the Mean-Field Approximation and incorporates the collective behavioral rules: cohesion, separation, and velocity alignment. We follow a distributed control scheme and show that our method can control a swarm of UAVs to a formation and velocity consensus with real-time collision avoidance. We validate the proposed method with physical and high-fidelity simulation experiments.Comment: To appear in the proceedings of IEEE International Conference on Robotics and Automation (ICRA), 202

Collaborative Control of Autonomous Swarms with Resource Constraints

This dissertation focuses on the collaborative control of homogeneous UAV swarms. A two-level scheme is proposed by combining the high-level path planning and the lowlevel vehicle motion control. A decentralized artificial potential function (APF) based approach, which mimics the bacteria foraging process, is studied for the high-level path planning. The deterministic potential based approach, however, suffers from the local minima entrapment dilemma, which motivate us to fix the "flaw" that is naturally embedded. An innovative decentralized stochastic approach based on the Markov Random Filed (MRF) theory is proposed; this approach traditionally used in statistical mechanics and in image processing. By modeling the local interactions as Gibbs potentials, the movements of vehicles are then decided by using Gibbs sampler based simulated annealing (SA) algorithm. A two-step sampling scheme is proposed to coordinate vehicle networks: in the first sampling step, a vehicle is picked through a properly designed, configuration-dependent proposal distribution, and in the second sampling step, the vehicle makes a move by using the local characteristics of the Gibbs distribution. Convergence properties are established theoretically and confirmed with simulations. In order to reduce the communication cost and the delay, a fully parallel sampling algorithm is studied and analyzed accordingly. In practice, the stochastic nature of the proposed algorithm might lead to a high traveling cost. To mitigate this problem, a hybrid algorithm is eveloped by combining the Gibbs sampler based method with the deterministic gradient-flow method to gain the advantages of both approaches. The robustness of the Gibbs sampler based algorithm is also studied. The convergence properties are investigated for different types sensor errors including range-error and random-error. Error bounds are derived to guarantee the convergence of the stochastic algorithm. In the low-level motion control module, a model predictive control (MPC) approach is investigated for car-like UAV model. Multiple control objectives, for example, minimizing tracking error, avoiding actuator/state saturation, and minimizing control effort, are easily encoded in the objective function. Two numerical optimization approaches, gradient descendent approach and dynamic programming approach, are studied to strike the balance between computation time and complexity

Throughput Maximization in Unmanned Aerial Vehicle Networks

The use of Unmanned Aerial Vehicles (UAVs) swarms in civilian applications such as surveillance, agriculture, search and rescue, and border patrol is becoming popular. UAVs have also found use as mobile or portable base stations. In these applications, communication requirements for UAVs are generally stricter as compared to conventional aircrafts. Hence, there needs to be an efficient Medium Access Control (MAC) protocol that ensures UAVs experience low channel access delays and high throughput. Some challenges when designing UAVs MAC protocols include interference and rapidly changing channel states, which require a UAV to adapt its data rate to ensure data transmission success. Other challenges include Quality of Service (QoS) requirements and multiple contending UAVs that result in collisions and channel access delays. To this end, this thesis aims to utilize Multi-Packet Reception (MPR) technology. In particular, it considers nodes that are equipped with a Successive Interference Cancellation (SIC) radio, and thereby, allowing them to receive multiple transmissions simultaneously. A key problem is to identify a suitable a Time Division Multiple Access (TDMA) transmission schedule that allows UAVs to transmit successfully and frequently. Moreover, in order for SIC to operate, there must be a sufficient difference in received power. However, in practice, due to the location and orientation of nodes, the received power of simultaneously transmitting nodes may cause SIC decoding to fail at a receiver. Consequently, a key problem concerns the placement and orientation of UAVs to ensure there is diversity in received signal strength at a receiving node. Lastly, interference between UAVs serving as base station is a critical issue. In particular, their respective location may have excessive interference or cause interference to other UAVs; all of which have an impact on the schedule used by these UAVs to serve their respective users

A Distributed Learning Algorithm with Bit-valued Communications for Multi-agent Welfare Optimization

A multi-agent system comprising N agents, each picking actions from a finite set and receiving a payoff that depends on the action of the whole, is considered. The exact form of the payoffs are unknown and only their values can be measured by the respective agents. A decentralized algorithm was proposed by Marden et. al. [1] and in the authors’ earlier work [2] that, in this setting, leads to the agents picking welfare optimizing actions under some restrictive assumptions on the payoff structure. This algorithm is modified in this paper to incorporate exchange of certain bit-valued information between the agents over a directed communication graph. The notion of an interaction graph is then introduced to encode known interaction in the system. Restrictions on the payoff structure are eliminated and conditions that guarantee convergence to welfare minimizing actions w.p. 1 are derived under the assumption that the union of the interaction graph and communication graph is strongly connected.Research partially supported by the US Air Force Office of Scientific Research MURI grant FA9550-09-1-0538 and by the National Science Foundation (NSF) grant CNS-1035655

Design and Performance Analysis of Genetic Algorithms for Topology Control Problems

In this dissertation, we present a bio-inspired decentralized topology control mechanism, called force-based genetic algorithm (FGA), where a genetic algorithm (GA) is run by each autonomous mobile node to achieve a uniform spread of mobile nodes and to provide a fully connected network over an unknown area. We present a formal analysis of FGA in terms of convergence speed, uniformity at area coverage, and Lyapunov stability theorem. This dissertation emphasizes the use of mobile nodes to achieve a uniform distribution over an unknown terrain without a priori information and a central control unit. In contrast, each mobile node running our FGA has to make its own movement direction and speed decisions based on local neighborhood information, such as obstacles and the number of neighbors, without a centralized control unit or global knowledge. We have implemented simulation software in Java and developed four different testbeds to study the effectiveness of different GA-based topology control frameworks for network performance metrics including node density, speed, and the number of generations that GAs run. The stochastic behavior of FGA, like all GA-based approaches, makes it difficult to analyze its convergence speed. We built metrically transitive homogeneous and inhomogeneous Markov chain models to analyze the convergence of our FGA with respect to the communication ranges of mobile nodes and the total number of nodes in the system. The Dobrushin contraction coefficient of ergodicity is used for measuring convergence speed for homogeneous and inhomogeneous Markov chain models of our FGA. Furthermore, convergence characteristic analysis helps us to choose the nearoptimal values for communication range, the number of mobile nodes, and the mean node degree before sending autonomous mobile nodes to any mission. Our analytical and experimental results show that our FGA delivers promising results for uniform mobile node distribution over unknown terrains. Since our FGA adapts to local environment rapidly and does not require global network knowledge, it can be used as a real-time topology controller for commercial and military applications

Consensus problems and the effects of graph topology in collaborative control

In this dissertation, several aspects of design for networked systems are addressed. The main focus is on combining approaches from system theory and graph theory to characterize graph topologies that result in efficient decision making and control. In this framework, modelling and design of sparse graphs that are robust to failures and provide high connectivity are considered. A decentralized approach to path generation in a collaborative system is modelled using potential functions. Taking inspiration from natural swarms, various behaviors of the system such as target following, moving in cohesion and obstacle avoidance are addressed by appropriate encoding of the corresponding costs in the potential function and using gradient descent for minimizing the energy function. Different emergent behaviors emerge as a result of varying the weights attributed with different components of the potential function. Consensus problems are addressed as a unifying theme in many collaborative control problems and their robustness and convergence properties are studied. Implications of the continuous convergence property of consensus problems on their reachability and robustness are studied. The effects of link and agent faults on consensus problems are also investigated. In particular the concept of invariant nodes has been introduced to model the effect of nodes with different behaviors from regular nodes. A fundamental association is established between the structural properties of a graph and the performance of consensus algorithms running on them. This leads to development of a rigorous evaluation of the topology effects and determination of efficient graph topologies. It is well known that graphs with large diameter are not efficient as far as the speed of convergence of distributed algorithms is concerned. A challenging problem is to determine a minimum number of long range links (shortcuts), which guarantees a level of enhanced performance. This problem is investigated here in a stochastic framework. Specifically, the small world model of Watts and Strogatz is studied and it is shown that adding a few long range edges to certain graph topologies can significantly increase both the rate of convergence for consensus algorithms and the number of spanning trees in the graph. The simulations are supported by analytical stochastic methods inspired from perturbations of Markov chains. This approach is further extended to a probabilistic framework for understanding and quantifying the small world effect on consensus convergence rates: Time varying topologies, in which each agent nominally communicates according to a predefined topology, and switching with non-neighboring agents occur with small probability is studied. A probabilistic framework is provided along with fundamental bounds on the convergence speed of consensus problems with probabilistic switching. The results are also extended to the design of robust topologies for distributed algorithms. The design of a semi-distributed two-level hierarchical network is also studied, leading to improvement in the performance of distributed algorithms. The scheme is based on the concept of social degree and local leader selection and the use of consensus-type algorithms for locally determining topology information. Future suggestions include adjusting our algorithm towards a fully distributed implementation. Another important aspect of performance in collaborative systems is for the agents to send and receive information in a manner that minimizes process costs, such as estimation error and the cost of control. An instance of this problem is addressed by considering a collaborative sensor scheduling problem. It is shown that in finding the optimal joint estimates, the general tree-search solution can be efficiently solved by devising a method that utilizes the limited processing capabilities of agents to significantly decrease the number of search hypotheses

Learning Models of Behavior From Demonstration and Through Interaction

This dissertation is concerned with the autonomous learning of behavioral models for sequential decision-making. It addresses both the theoretical aspects of behavioral modeling — like the learning of appropriate task representations — and the practical difficulties regarding algorithmic implementation. The first half of the dissertation deals with the problem of learning from demonstration, which consists in generalizing the behavior of an expert demonstrator based on observation data. Two alternative modeling paradigms are discussed. First, a nonparametric inference framework is developed to capture the behavior of the expert at the policy level. A key challenge in the design of the framework is the objective of making minimal assumptions about the observed behavior type while dealing with a potentially infinite number of system states. Due to the automatic adaptation of the model order to the complexity of the shown behavior, the proposed approach is able to pick up stochastic expert policies of arbitrary structure. Second, a nonparametric inverse reinforcement learning framework based on subgoal modeling is proposed, which allows to efficiently reconstruct the expert behavior at the intentional level. Other than most existing approaches, the proposed methodology naturally handles periodic tasks and situations where the intentions of the expert change over time. By adaptively decomposing the decision-making problem into a series of task-related subproblems, both inference frameworks are suitable for learning compact encodings of the expert behavior. For performance evaluation, the models are compared with existing frameworks on synthetic benchmark scenarios and real-world data recorded on a KUKA lightweight robotic arm. In the second half of the work, the focus shifts to multi-agent modeling, with the aim of analyzing the decision-making process in large-scale homogeneous agent networks. To fill the gap of decentralized system models with explicit agent homogeneity, a new class of agent systems is introduced. For this system class, the problem of inverse reinforcement learning is discussed and a meta learning algorithm is devised that makes explicit use of the system symmetries. As part of the algorithm, a heterogeneous reinforcement learning scheme is proposed for optimizing the collective behavior of the system based on the local state observations made at the agent level. Finally, to scale the simulation of the network to large agent numbers, a continuum version of the model is derived. After discussing the system components and associated optimality criteria, numerical examples of collective tasks are given that demonstrate the capabilities of the continuum approach and show its advantages over large-scale agent-based modeling

A review of source term estimation methods for atmospheric dispersion events using static or mobile sensors

Understanding atmospheric transport and dispersal events has an important role in a range of scenarios. Of particular importance is aiding in emergency response after an intentional or accidental chemical, biological or radiological (CBR) release. In the event of a CBR release, it is desirable to know the current and future spatial extent of the contaminant as well as its location in order to aid decision makers in emergency response. Many dispersion phenomena may be opaque or clear, thus monitoring them using visual methods will be difficult or impossible. In these scenarios, relevant concentration sensors are required to detect the substance where they can form a static network on the ground or be placed upon mobile platforms. This paper presents a review of techniques used to gain information about atmospheric dispersion events using static or mobile sensors. The review is concluded with a discussion on the current limitations of the state of the art and recommendations for future research

Multi-Robot Symbolic Task and Motion Planning Leveraging Human Trust Models: Theory and Applications

Multi-robot systems (MRS) can accomplish more complex tasks with two or more robots and have produced a broad set of applications. The presence of a human operator in an MRS can guarantee the safety of the task performing, but the human operators can be subject to heavier stress and cognitive workload in collaboration with the MRS than the single robot. It is significant for the MRS to have the provable correct task and motion planning solution for a complex task. That can reduce the human workload during supervising the task and improve the reliability of human-MRS collaboration. This dissertation relies on formal verification to provide the provable-correct solution for the robotic system. One of the challenges in task and motion planning under temporal logic task specifications is developing computationally efficient MRS frameworks. The dissertation first presents an automaton-based task and motion planning framework for MRS to satisfy finite words of linear temporal logic (LTL) task specifications in parallel and concurrently. Furthermore, the dissertation develops a computational trust model to improve the human-MRS collaboration for a motion task. Notably, the current works commonly underemphasize the environmental attributes when investigating the impacting factors of human trust in robots. Our computational trust model builds a linear state-space (LSS) equation to capture the influence of environment attributes on human trust in an MRS. A Bayesian optimization based experimental design (BOED) is proposed to sequentially learn the human-MRS trust model parameters in a data-efficient way. Finally, the dissertation shapes a reward function for the human-MRS collaborated complex task by referring to the above LTL task specification and computational trust model. A Bayesian active reinforcement learning (RL) algorithm is used to concurrently learn the shaped reward function and explore the most trustworthy task and motion planning solution