Probabilistic Plan Synthesis for Coupled Multi-Agent Systems

This paper presents a fully automated procedure for controller synthesis for multi-agent systems under the presence of uncertainties. We model the motion of each of the $N$ agents in the environment as a Markov Decision Process (MDP) and we assign to each agent one individual high-level formula given in Probabilistic Computational Tree Logic (PCTL). Each agent may need to collaborate with other agents in order to achieve a task. The collaboration is imposed by sharing actions between the agents. We aim to design local control policies such that each agent satisfies its individual PCTL formula. The proposed algorithm builds on clustering the agents, MDP products construction and controller policies design. We show that our approach has better computational complexity than the centralized case, which traditionally suffers from very high computational demands.Comment: IFAC WC 2017, Toulouse, Franc

Scalable Approach to Uncertainty Quantification and Robust Design of Interconnected Dynamical Systems

Development of robust dynamical systems and networks such as autonomous aircraft systems capable of accomplishing complex missions faces challenges due to the dynamically evolving uncertainties coming from model uncertainties, necessity to operate in a hostile cluttered urban environment, and the distributed and dynamic nature of the communication and computation resources. Model-based robust design is difficult because of the complexity of the hybrid dynamic models including continuous vehicle dynamics, the discrete models of computations and communications, and the size of the problem. We will overview recent advances in methodology and tools to model, analyze, and design robust autonomous aerospace systems operating in uncertain environment, with stress on efficient uncertainty quantification and robust design using the case studies of the mission including model-based target tracking and search, and trajectory planning in uncertain urban environment. To show that the methodology is generally applicable to uncertain dynamical systems, we will also show examples of application of the new methods to efficient uncertainty quantification of energy usage in buildings, and stability assessment of interconnected power networks

Sampling-based Synthesis of Controllers for Multiple Agents under Signal Temporal Logic Specifications

openL’ampia applicazione dei robot nelle industrie e nella società ha portato alla necessità di prescrivere complessi compiti di alto livello ad agenti autonomi. Signal Temporal Logic (STL) è una logica temporale che consente di esprimere requisiti spazio-temporali e quantificare il livello di soddisfazione delle preferenze. Quando si pianifica considerando specifiche STL, la sfida principale è generare traiettorie che soddisfino le formule logiche e seguire le traiettorie così ottenute. Il progetto propone una soluzione per il problema di pianificazione del movimento di multipli agenti autonomi, soggetti a specifiche STL accoppiate. Partendo da uno scenario in cui sono coinvolti solo due agenti, un algoritmo basato sul campionamento, Coupled STL_RRT*, è progettato. L’approccio proposto, basato su RRT*, costruisce in modo distribuito due alberi nel dominio del tempo e dello stato accoppiati. Per ogni sistema dinamico, data una posizione iniziale, la strategia sviluppata trova la traiettoria probabilisticamente ottimale in termini di una funzione di costo che dipende dagli input di controllo richiesti. Prima di aggiungere nuovi stati all’albero corrispondente, l’algoritmo controlla se la formula logica non viene violata, assicurando quindi che la traiettoria finale, variabile nel tempo, soddisfi le specifiche spazio-temporali. La dinamica dell’agente autonomo è presa direttamente in considerazione e il concetto di raggiungibilità viene sfruttato per ottenere traiettorie ammissibili rispetto ai vincoli dinamici. L’algoritmo è quindi simulato, considerando un ambiente con ostacoli statici e diversi requisiti STL, specificati dall’utente. L’approccio viene poi esteso al caso di sistemi multi-agente con più di tre agenti. Come nel caso precedente, l’algoritmo costruisce un albero spazio-temporale per ciascun agente, assicurando che la traiettoria finale soddisfi i requisiti STL. La soluzione proposta è poi verificata in scenari simulati, considerando sistemi con 4 o 6 agenti.The wide application of robots in industries and society has brought the need to prescribe complex high-level tasks to autonomous agents. Signal Temporal Logic (STL) is a temporal logic that allows to express desired spatio-temporal requirements, while quantifying the satisfaction of the preferences. When planning under STL specifications, the main challenge is to generate trajectories that satisfy the logical formulas and to track those trajectories. The project proposes a solution for the motion planning problem of multiple autonomous agents, subject to coupled STL specifications. Starting from a scenario where only two agents are involved, a sampling-based algorithm, Coupled STL_RRT*, is designed. The proposed RRT*-based approach builds two trees in the coupled time and state domain in a distributed manner. For each dynamical system, given an initial position, the developed strategy finds a probabilistic optimal trajectory in terms of a cost function that depends on the required control inputs. Before adding new states to the corresponding tree, the algorithm checks if the logical formula is not violated, hence ensuring that the final time-varying trajectory satisfies the spatio-temporal specifications. The dynamics of the autonomous agent is directly taken into account and reachability is exploited to obtain a trajectory that is feasible with respect to the dynamic constraints. The algorithm is then simulated, considering an environment with static obstacles and different STL requirements, specified by the user. The approach is then extended to the case of multi-agent systems with more than three agents. As in the previous case, the algorithm builds a spatiotemporal tree for each agent, ensuring that the final trajectory satisfies the STL requirements. The proposed solution is then verified in simulated scenarios, considering 4-agents and 6-agents systems

Planning for Decentralized Control of Multiple Robots Under Uncertainty

We describe a probabilistic framework for synthesizing control policies for general multi-robot systems, given environment and sensor models and a cost function. Decentralized, partially observable Markov decision processes (Dec-POMDPs) are a general model of decision processes where a team of agents must cooperate to optimize some objective (specified by a shared reward or cost function) in the presence of uncertainty, but where communication limitations mean that the agents cannot share their state, so execution must proceed in a decentralized fashion. While Dec-POMDPs are typically intractable to solve for real-world problems, recent research on the use of macro-actions in Dec-POMDPs has significantly increased the size of problem that can be practically solved as a Dec-POMDP. We describe this general model, and show how, in contrast to most existing methods that are specialized to a particular problem class, it can synthesize control policies that use whatever opportunities for coordination are present in the problem, while balancing off uncertainty in outcomes, sensor information, and information about other agents. We use three variations on a warehouse task to show that a single planner of this type can generate cooperative behavior using task allocation, direct communication, and signaling, as appropriate

Service composition in stochastic settings

With the growth of the Internet-of-Things and online Web services, more services with more capabilities are available to us. The ability to generate new, more useful services from existing ones has been the focus of much research for over a decade. The goal is, given a specification of the behavior of the target service, to build a controller, known as an orchestrator, that uses existing services to satisfy the requirements of the target service. The model of services and requirements used in most work is that of a finite state machine. This implies that the specification can either be satisfied or not, with no middle ground. This is a major drawback, since often an exact solution cannot be obtained. In this paper we study a simple stochastic model for service composition: we annotate the tar- get service with probabilities describing the likelihood of requesting each action in a state, and rewards for being able to execute actions. We show how to solve the resulting problem by solving a certain Markov Decision Process (MDP) derived from the service and requirement specifications. The solution to this MDP induces an orchestrator that coincides with the exact solution if a composition exists. Otherwise it provides an approximate solution that maximizes the expected sum of values of user requests that can be serviced. The model studied although simple shades light on composition in stochastic settings and indeed we discuss several possible extensions