Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search

Today's automated vehicles lack the ability to cooperate implicitly with others. This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments. Based on cooperative modeling of other agents and Decoupled-UCT (a variant of MCTS), the algorithm evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants. Macro-actions allow for temporal extension over multiple time steps and increase the effective search depth requiring fewer iterations to plan over longer horizons. Without predefined policies for macro-actions, the algorithm simultaneously learns policies over and within macro-actions. The proposed method is evaluated under several conflict scenarios, showing that the algorithm can achieve effective cooperative planning with learned macro-actions in heterogeneous environments

Dynamic matching and bargaining games: A general approach

This paper presents a new characterization result for competitive allocations in quasilinear economies. This result is informed by the analysis of non-cooperative dynamic search and bargaining games. Such games provide models of decentralized markets with trading frictions. A central objective of this literature is to investigate how equilibrium outcomes depend on the level of the frictions. In particular, does the trading outcome become Walrasian when frictions become small? Existing specifications of such games provide divergent answers. The characterization result is used to investigate what causes these differences and to generalize insights from the analysis of specific search and bargaining games.Dynamic Matching and Bargaining, Decentralized Markets, Non-cooperative Foundations of Competitive Equilibrium, Search Theory

The provision of quality in a bilateral search market

We accomplish two goals. First, we provide a non-cooperative foundation for the use of the Nash bargaining solution in search markets. This finding should help to close the rift between the search and the matching-and-bargaining literature. Second, we establish that the diversity of quality offered (at an increasing price-quality ratio) in a decentralized market is an equilibrium phenomenon - even in the limit as search frictions disappear.quality dispersion, Nash Program, bilateral search

The provision of quality in a bilateral search market

We accomplish two goals. First, we provide a non-cooperative foundation for the use of the Nash bargaining solution in search markets. This finding should help to close the rift between the search and the matching-and-bargaining literature. Second, we establish that the diversity of quality offered (at an increasing price-quality ratio) in a decentralized market is an equilibrium phenomenon -even in the limit as search frictions disappear.quality dispersion, Nash Program, bilateral search.

Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search

Urban traffic scenarios often require a high degree of cooperation between traffic participants to ensure safety and efficiency. Observing the behavior of others, humans infer whether or not others are cooperating. This work aims to extend the capabilities of automated vehicles, enabling them to cooperate implicitly in heterogeneous environments. Continuous actions allow for arbitrary trajectories and hence are applicable to a much wider class of problems than existing cooperative approaches with discrete action spaces. Based on cooperative modeling of other agents, Monte Carlo Tree Search (MCTS) in conjunction with Decoupled-UCT evaluates the action-values of each agent in a cooperative and decentralized way, respecting the interdependence of actions among traffic participants. The extension to continuous action spaces is addressed by incorporating novel MCTS-specific enhancements for efficient search space exploration. The proposed algorithm is evaluated under different scenarios, showing that the algorithm is able to achieve effective cooperative planning and generate solutions egocentric planning fails to identify

MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPs

We present multi-agent A* (MAA*), the first complete and optimal heuristic search algorithm for solving decentralized partially-observable Markov decision problems (DEC-POMDPs) with finite horizon. The algorithm is suitable for computing optimal plans for a cooperative group of agents that operate in a stochastic environment such as multirobot coordination, network traffic control, `or distributed resource allocation. Solving such problems efiectively is a major challenge in the area of planning under uncertainty. Our solution is based on a synthesis of classical heuristic search and decentralized control theory. Experimental results show that MAA* has significant advantages. We introduce an anytime variant of MAA* and conclude with a discussion of promising extensions such as an approach to solving infinite horizon problems.Comment: Appears in Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (UAI2005

Cooperative Control Simulation Validation Using Applied Probability Theory

Several research simulations have been created to support development and refinement of teamed autonomous agents using decentralized cooperative control algorithms. Simulation is the necessary tool to evaluate the performance of decentralized cooperative control algorithms, however these simulations lack a method to validate their output. This research presents a method to validate the performance of a decentralized cooperative control simulation environment for an autonomous Wide Area Search Munition (WASM). Rigorous analytical methods for six wide area search and engagement scenarios involving Uniform, Normal, and Poisson distributions of N real targets and M false target objects are formulated to generate expected numbers of target attacks and kills for a searching WASM. The mean value based on the number of target attack and kills from Monte Carlo simulations representative of the individual scenarios are compared to the analytically derived expected values. Emphasis is placed on Wide Area Search Munitions (WASMs) operating in a multiple target environment where a percentage of the total targets are either false targets or may be misconstrued as false by varying the capability of the WASM’s Automatic Target Recognition (ATR) capability

The provision of quality in a bilateral search market

We accomplish two goals. First, we provide a non-cooperative foundation for the use of the Nash bargaining solution in search markets. This finding should help to close the rift between the search and the matching-and-bargaining literature. Second, we establish that the diversity of quality offered (at an increasing price-quality ratio) in a decentralized market is an equilibrium phenomenon - even in the limit as search frictions disappear

CODEA : an agent based multi-objective optimization framework

This work presents CODEA, a COoperative DEcentralized Architecture for Multiobjective Optimization. CODEA is an objectoriented framework that aims at the creation of groups of agents to tackle complex problems by cooperative search. This cooperation is carried out without any individual controlling the cooperation nor the behaviour of the agents. Each agent works on its own to improve itself and collaborates to improve the performance of the group by sharing information