An Auction-based Coordination Strategy for Task-Constrained Multi-Agent Stochastic Planning with Submodular Rewards

In many domains such as transportation and logistics, search and rescue, or cooperative surveillance, tasks are pending to be allocated with the consideration of possible execution uncertainties. Existing task coordination algorithms either ignore the stochastic process or suffer from the computational intensity. Taking advantage of the weakly coupled feature of the problem and the opportunity for coordination in advance, we propose a decentralized auction-based coordination strategy using a newly formulated score function which is generated by forming the problem into task-constrained Markov decision processes (MDPs). The proposed method guarantees convergence and at least 50% optimality in the premise of a submodular reward function. Furthermore, for the implementation on large-scale applications, an approximate variant of the proposed method, namely Deep Auction, is also suggested with the use of neural networks, which is evasive of the troublesome for constructing MDPs. Inspired by the well-known actor-critic architecture, two Transformers are used to map observations to action probabilities and cumulative rewards respectively. Finally, we demonstrate the performance of the two proposed approaches in the context of drone deliveries, where the stochastic planning for the drone league is cast into a stochastic price-collecting Vehicle Routing Problem (VRP) with time windows. Simulation results are compared with state-of-the-art methods in terms of solution quality, planning efficiency and scalability.Comment: 17 pages, 5 figure

Decentralized task allocation for multiple UAVs with task execution uncertainties

This work builds on a robust decentralized task allocation algorithm to address the multiple unmanned aerial vehicle (UAV) surveillance problem under task duration uncertainties. Considering the existing robust task allocation algorithm is computationally intensive and also has no optimality guarantees, this paper proposes a new robust task assignment formulation that reduces the calculation of robust scores and provides a certain theoretical guarantee of optimality. In the proposed method, the Markov model is introduced to describe the impact of uncertain parameters on task rewards and the expected score function is reformulated as the utility function of the states in the Markov model. Through providing the high-precision expected marginal gain of tasks, the task assignment gains a better accumulative score than the state of arts robust algorithms do. Besides, this algorithm is proven to be convergent and could reach a prior optimality guarantee of at least 50%. Numerical Simulations demonstrate the performance improvement of the proposed method compared with basic CBBA, robust extension to CBBA and cost-benefit greedy algorithm

Multiagent planning with Bayesian nonparametric asymptotics

Thesis: S.M., Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (pages 95-105).Autonomous multiagent systems are beginning to see use in complex, changing environments that cannot be completely specified a priori. In order to be adaptive to these environments and avoid the fragility associated with making too many a priori assumptions, autonomous systems must incorporate some form of learning. However, learning techniques themselves often require structural assumptions to be made about the environment in which a system acts. Bayesian nonparametrics, on the other hand, possess structural flexibility beyond the capabilities of past parametric techniques commonly used in planning systems. This extra flexibility comes at the cost of increased computational cost, which has prevented the widespread use of Bayesian nonparametrics in realtime autonomous planning systems. This thesis provides a suite of algorithms for tractable, realtime, multiagent planning under uncertainty using Bayesian nonparametrics. The first contribution is a multiagent task allocation framework for tasks specified as Markov decision processes. This framework extends past work in multiagent allocation under uncertainty by allowing exact distribution propagation instead of sampling, and provides an analytic solution time/quality tradeoff for system designers. The second contribution is the Dynamic Means algorithm, a novel clustering method based upon Bayesian nonparametrics for realtime, lifelong learning on batch-sequential data containing temporally evolving clusters. The relationship with previous clustering models yields a modelling scheme that is as fast as typical classical clustering approaches while possessing the flexibility and representational power of Bayesian nonparametrics. The final contribution is Simultaneous Clustering on Representation Expansion (SCORE), which is a tractable model-based reinforcement learning algorithm for multimodel planning problems, and serves as a link between the aforementioned task allocation framework and the Dynamic Means algorithmby Trevor D. J. Campbell.S.M

Active Object Classification from 3D Range Data with Mobile Robots

This thesis addresses the problem of how to improve the acquisition of 3D range data with a mobile robot for the task of object classification. Establishing the identities of objects in unknown environments is fundamental for robotic systems and helps enable many abilities such as grasping, manipulation, or semantic mapping. Objects are recognised by data obtained from sensor observations, however, data is highly dependent on viewpoint; the variation in position and orientation of the sensor relative to an object can result in large variation in the perception quality. Additionally, cluttered environments present a further challenge because key data may be missing. These issues are not always solved by traditional passive systems where data are collected from a fixed navigation process then fed into a perception pipeline. This thesis considers an active approach to data collection by deciding where is most appropriate to make observations for the perception task. The core contributions of this thesis are a non-myopic planning strategy to collect data efficiently under resource constraints, and supporting viewpoint prediction and evaluation methods for object classification. Our approach to planning uses Monte Carlo methods coupled with a classifier based on non-parametric Bayesian regression. We present a novel anytime and non-myopic planning algorithm, Monte Carlo active perception, that extends Monte Carlo tree search to partially observable environments and the active perception problem. This is combined with a particle-based estimation process and a learned observation likelihood model that uses Gaussian process regression. To support planning, we present 3D point cloud prediction algorithms and utility functions that measure the quality of viewpoints by their discriminatory ability and effectiveness under occlusion. The utility of viewpoints is quantified by information-theoretic metrics, such as mutual information, and an alternative utility function that exploits learned data is developed for special cases. The algorithms in this thesis are demonstrated in a variety of scenarios. We extensively test our online planning and classification methods in simulation as well as with indoor and outdoor datasets. Furthermore, we perform hardware experiments with different mobile platforms equipped with different types of sensors. Most significantly, our hardware experiments with an outdoor robot are to our knowledge the first demonstrations of online active perception in a real outdoor environment. Active perception has broad significance in many applications. This thesis emphasises the advantages of an active approach to object classification and presents its assimilation with a wide range of robotic systems, sensors, and perception algorithms. By demonstration of performance enhancements and diversity, our hope is that the concept of considering perception and planning in an integrated manner will be of benefit in improving current systems that rely on passive data collection

Decentralized task allocation for dynamic, time-sensitive tasks

Thesis: S.M., Massachusetts Institute of Technology, Department of Mechanical Engineering, 2018.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student-submitted PDF version of thesis.Includes bibliographical references (pages 103-110).In time-sensitive and dynamic missions, autonomous vehicles must respond quickly to new information and objectives. In the case of dynamic task allocation, a team of agents are presented with a new, unknown task that must be allocated with their original allocations. This is exacerbated further in decentralized settings where agents are limited to utilizing local information during the allocation process. This thesis presents a fully decentralized, dynamic task allocation algorithm that extends the Consensus-Based Bundle Algorithm (CBBA) to allow for allocating new tasks. Whereas static CBBA requires a full resetting of previous allocations, CBBA with Partial Replanning (CBBA-PR) enables the agents to only partially reset their allocations to efficiently and quickly allocate a new task. By varying the number of existing tasks that are reset during replan, the team can trade-off convergence speed with amount of coordination. By specifically choosing the lowest bid tasks for resetting, CBBA-PR is shown to converge linearly with the number of tasks reset and the network diameter of the team. In addition, limited replanning methods are presented for scenarios without sufficient replanning time. These include a single reset bidding procedure for agents at capacity, a no-replanning heuristic that can identify scenarios that does not require replanning, and a subteam formation algorithm for reducing the network diameter. Finally, this thesis describes hardware and simulation experiments used to explore the effects of ad-hoc, decentralized communication on consensus algorithms and to validate the performance of CBBA-PR.by Noam Buckman.S.M

Emergency rapid mapping with drones: models and solution approaches for offline and online mission planning

Die Verfügbarkeit von unbemannten Luftfahrzeugen (unmanned aerial vehicles oder UAVs) und die Fortschritte in der Entwicklung leichtgewichtiger Sensorik eröffnen neue Möglichkeiten für den Einsatz von Fernerkundungstechnologien zur Schnellerkundung in Großschadenslagen. Hier ermöglichen sie es beispielsweise nach Großbränden, Einsatzkräften in kurzer Zeit ein erstes Lagebild zur Verfügung zu stellen. Die begrenzte Flugdauer der UAVs wie auch der Bedarf der Einsatzkräfte nach einer schnellen Ersteinschätzung bedeuten jedoch, dass die betroffenen Gebiete nur stichprobenartig überprüft werden können. In Kombination mit Interpolationsverfahren ermöglichen diese Stichproben anschließend eine Abschätzung der Verteilung von Gefahrstoffen. Die vorliegende Arbeit befasst sich mit dem Problem der Planung von UAV-Missionen, die den Informationsgewinn im Notfalleinsatz maximieren. Das Problem wird dabei sowohl in der Offline-Variante, die Missionen vor Abflug bestimmt, als auch in der Online-Variante, bei der die Pläne während des Fluges der UAVs aktualisiert werden, untersucht. Das übergreifende Ziel ist die Konzeption effizienter Modelle und Verfahren, die Informationen über die räumliche Korrelation im beobachteten Gebiet nutzen, um in zeitkritischen Situationen Lösungen von hoher Vorhersagegüte zu bestimmen. In der Offline-Planung wird das generalized correlated team orienteering problem eingeführt und eine zweistufige Heuristik zur schnellen Bestimmung explorativer UAV-Missionen vorgeschlagen. In einer umfangreichen Studie wird die Leistungsfähigkeit und Konkurrenzfähigkeit der Heuristik hinsichtlich Rechenzeit und Lösungsqualität bestätigt. Anhand von in dieser Arbeit neu eingeführten Benchmarkinstanzen wird der höhere Informationsgewinn der vorgeschlagenen Modelle im Vergleich zu verwandten Konzepten aufgezeigt. Im Bereich der Online-Planung wird die Kombination von lernenden Verfahren zur Modellierung der Schadstoffe mit Planungsverfahren, die dieses Wissen nutzen, um Missionen zu verbessern, untersucht. Hierzu wird eine breite Spanne von Lösungsverfahren aus unterschiedlichen Disziplinen klassifiziert und um neue effiziente Modellierungsvarianten für die Schnellerkundung ergänzt. Die Untersuchung im Rahmen einer ereignisdiskreten Simulation zeigt, dass vergleichsweise einfache Approximationen räumlicher Zusammenhänge in sehr kurzer Zeit Lösungen hoher Qualität ermöglichen. Darüber hinaus wird die höhere Robustheit genauerer, aber aufwändigerer Modelle und Lösungskonzepte demonstriert

Supply Chain

Traditionally supply chain management has meant factories, assembly lines, warehouses, transportation vehicles, and time sheets. Modern supply chain management is a highly complex, multidimensional problem set with virtually endless number of variables for optimization. An Internet enabled supply chain may have just-in-time delivery, precise inventory visibility, and up-to-the-minute distribution-tracking capabilities. Technology advances have enabled supply chains to become strategic weapons that can help avoid disasters, lower costs, and make money. From internal enterprise processes to external business transactions with suppliers, transporters, channels and end-users marks the wide range of challenges researchers have to handle. The aim of this book is at revealing and illustrating this diversity in terms of scientific and theoretical fundamentals, prevailing concepts as well as current practical applications

Problems in Control, Estimation, and Learning in Complex Robotic Systems

In this dissertation, we consider a range of different problems in systems, control, and learning theory and practice. In Part I, we look at problems in control of complex networks. In Chapter 1, we consider the performance analysis of a class of linear noisy dynamical systems. In Chapter 2, we look at the optimal design problems for these networks. In Chapter 3, we consider dynamical networks where interactions between the networks occur randomly in time. And in the last chapter of this part, in Chapter 4, we look at dynamical networks wherein coupling between the subsystems (or agents) changes nonlinearly based on the difference between the state of the subsystems. In Part II, we consider estimation problems wherein we deal with a large body of variables (i.e., at large scale). This part starts with Chapter 5, in which we consider the problem of sampling from a dynamical network in space and time for initial state recovery. In Chapter 6, we consider a similar problem with the difference that the observations instead of point samples become continuous observations that happen in Lebesgue measurable observations. In Chapter 7, we consider an estimation problem in which the location of a robot during the navigation is estimated using the information of a large number of surrounding features and we would like to select the most informative features using an efficient algorithm. In Part III, we look at active perception problems, which are approached using reinforcement learning techniques. This part starts with Chapter 8, in which we tackle the problem of multi-agent reinforcement learning where the agents communicate and classify as a team. In Chapter 9, we consider a single agent version of the same problem, wherein a layered architecture replaces the architectures of the previous chapter. Then, we use reinforcement learning to design the meta-layer (to select goals), action-layer (to select local actions), and perception-layer (to conduct classification)