A CASE FOR DOMAIN-INDEPENDENT DETERMINISTIC MULTIAGENT

The notion of planning using multiple agents has been around since the very beginning of planning itself. It has been approached from various viewpoints especially in the multiagent systems community. Recently, domain-independent multiagent planning has gained more attention also in the automated planning community. In this paper, we shortly present the current state of the art, question some aspects of the research field and discuss the rising challenges

An Abstract Framework for Non-Cooperative Multi-Agent Planning

[EN] In non-cooperative multi-agent planning environments, it is essential to have a system that enables the agents¿ strategic behavior. It is also important to consider all planning phases, i.e., goal allocation, strategic planning, and plan execution, in order to solve a complete problem. Currently, we have no evidence of the existence of any framework that brings together all these phases for non-cooperative multi-agent planning environments. In this work, an exhaustive study is made to identify existing approaches for the different phases as well as frameworks and different applicable techniques in each phase. Thus, an abstract framework that covers all the necessary phases to solve these types of problems is proposed. In addition, we provide a concrete instantiation of the abstract framework using different techniques to promote all the advantages that the framework can offer. A case study is also carried out to show an illustrative example of how to solve a non-cooperative multi-agent planning problem with the presented framework. This work aims to establish a base on which to implement all the necessary phases using the appropriate technologies in each of them and to solve complex problems in different domains of application for non-cooperative multi-agent planning settings.This work was partially funded by MINECO/FEDER RTI2018-095390-B-C31 project of the Spanish government. Jaume Jordan and Vicent Botti are funded by Universitat Politecnica de Valencia (UPV) PAID-06-18 project. Jaume Jordan is also funded by grant APOSTD/2018/010 of Generalitat Valenciana Fondo Social Europeo.Jordán, J.; Bajo, J.; Botti, V.; Julian Inglada, VJ. (2019). An Abstract Framework for Non-Cooperative Multi-Agent Planning. Applied Sciences. 9(23):1-18. https://doi.org/10.3390/app9235180S118923De Weerdt, M., & Clement, B. (2009). Introduction to planning in multiagent systems. Multiagent and Grid Systems, 5(4), 345-355. doi:10.3233/mgs-2009-0133Dunne, P. E., Kraus, S., Manisterski, E., & Wooldridge, M. (2010). Solving coalitional resource games. Artificial Intelligence, 174(1), 20-50. doi:10.1016/j.artint.2009.09.005Torreño, A., Onaindia, E., Komenda, A., & Štolba, M. (2018). Cooperative Multi-Agent Planning. ACM Computing Surveys, 50(6), 1-32. doi:10.1145/3128584Fikes, R. E., & Nilsson, N. J. (1971). Strips: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 2(3-4), 189-208. doi:10.1016/0004-3702(71)90010-5Hoffmann, J., & Nebel, B. (2001). The FF Planning System: Fast Plan Generation Through Heuristic Search. Journal of Artificial Intelligence Research, 14, 253-302. doi:10.1613/jair.855Dukeman, A., & Adams, J. A. (2017). Hybrid mission planning with coalition formation. Autonomous Agents and Multi-Agent Systems, 31(6), 1424-1466. doi:10.1007/s10458-017-9367-7Hadad, M., Kraus, S., Ben-Arroyo Hartman, I., & Rosenfeld, A. (2013). Group planning with time constraints. Annals of Mathematics and Artificial Intelligence, 69(3), 243-291. doi:10.1007/s10472-013-9363-9Guo, Y., Pan, Q., Sun, Q., Zhao, C., Wang, D., & Feng, M. (2019). Cooperative Game-based Multi-Agent Path Planning with Obstacle Avoidance*. 2019 IEEE 28th International Symposium on Industrial Electronics (ISIE). doi:10.1109/isie.2019.8781205v. Neumann, J. (1928). Zur Theorie der Gesellschaftsspiele. Mathematische Annalen, 100(1), 295-320. doi:10.1007/bf01448847Mookherjee, D., & Sopher, B. (1994). Learning Behavior in an Experimental Matching Pennies Game. Games and Economic Behavior, 7(1), 62-91. doi:10.1006/game.1994.1037Ochs, J. (1995). Games with Unique, Mixed Strategy Equilibria: An Experimental Study. Games and Economic Behavior, 10(1), 202-217. doi:10.1006/game.1995.1030Applegate, C., Elsaesser, C., & Sanborn, J. (1990). An architecture for adversarial planning. IEEE Transactions on Systems, Man, and Cybernetics, 20(1), 186-194. doi:10.1109/21.47820Sailer, F., Buro, M., & Lanctot, M. (2007). Adversarial Planning Through Strategy Simulation. 2007 IEEE Symposium on Computational Intelligence and Games. doi:10.1109/cig.2007.368082Willmott, S., Richardson, J., Bundy, A., & Levine, J. (2001). Applying adversarial planning techniques to Go. Theoretical Computer Science, 252(1-2), 45-82. doi:10.1016/s0304-3975(00)00076-1Nau, D. S., Au, T. C., Ilghami, O., Kuter, U., Murdock, J. W., Wu, D., & Yaman, F. (2003). SHOP2: An HTN Planning System. Journal of Artificial Intelligence Research, 20, 379-404. doi:10.1613/jair.1141Knuth, D. E., & Moore, R. W. (1975). An analysis of alpha-beta pruning. Artificial Intelligence, 6(4), 293-326. doi:10.1016/0004-3702(75)90019-3Vickrey, W. (1961). COUNTERSPECULATION, AUCTIONS, AND COMPETITIVE SEALED TENDERS. The Journal of Finance, 16(1), 8-37. doi:10.1111/j.1540-6261.1961.tb02789.xClarke, E. H. (1971). Multipart pricing of public goods. Public Choice, 11(1), 17-33. doi:10.1007/bf01726210Groves, T. (1973). Incentives in Teams. Econometrica, 41(4), 617. doi:10.2307/1914085Savaux, J., Vion, J., Piechowiak, S., Mandiau, R., Matsui, T., Hirayama, K., … Silaghi, M. (2016). DisCSPs with Privacy Recast as Planning Problems for Self-Interested Agents. 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI). doi:10.1109/wi.2016.0057Buzing, P., Mors, A. ter, Valk, J., & Witteveen, C. (2006). Coordinating Self-interested Planning Agents. Autonomous Agents and Multi-Agent Systems, 12(2), 199-218. doi:10.1007/s10458-005-6104-4Ter Mors, A., & Witteveen, C. (s. f.). Coordinating Non Cooperative Planning Agents: Complexity Results. IEEE/WIC/ACM International Conference on Intelligent Agent Technology. doi:10.1109/iat.2005.60Hrnčíř, J., Rovatsos, M., & Jakob, M. (2015). Ridesharing on Timetabled Transport Services: A Multiagent Planning Approach. Journal of Intelligent Transportation Systems, 19(1), 89-105. doi:10.1080/15472450.2014.941759Galuszka, A., & Swierniak, A. (2009). Planning in Multi-agent Environment Using Strips Representation and Non-cooperative Equilibrium Strategy. Journal of Intelligent and Robotic Systems, 58(3-4), 239-251. doi:10.1007/s10846-009-9364-4Rosenthal, R. W. (1973). A class of games possessing pure-strategy Nash equilibria. International Journal of Game Theory, 2(1), 65-67. doi:10.1007/bf01737559Jordán, J., Torreño, A., de Weerdt, M., & Onaindia, E. (2017). A better-response strategy for self-interested planning agents. Applied Intelligence, 48(4), 1020-1040. doi:10.1007/s10489-017-1046-5Veloso, M., Muñoz-Avila, H., & Bergmann, R. (1996). Case-based planning: selected methods and systems. AI Communications, 9(3), 128-137. doi:10.3233/aic-1996-9305VOORNEVELD, M., BORM, P., VAN MEGEN, F., TIJS, S., & FACCHINI, G. (1999). CONGESTION GAMES AND POTENTIALS RECONSIDERED. International Game Theory Review, 01(03n04), 283-299. doi:10.1142/s0219198999000219Han-Lim Choi, Brunet, L., & How, J. P. (2009). Consensus-Based Decentralized Auctions for Robust Task Allocation. IEEE Transactions on Robotics, 25(4), 912-926. doi:10.1109/tro.2009.2022423Monderer, D., & Shapley, L. S. (1996). Potential Games. Games and Economic Behavior, 14(1), 124-143. doi:10.1006/game.1996.0044Friedman, J. W., & Mezzetti, C. (2001). Learning in Games by Random Sampling. Journal of Economic Theory, 98(1), 55-84. doi:10.1006/jeth.2000.2694Aamodt, A., & Plaza, E. (1994). Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches. AI Communications, 7(1), 39-59. doi:10.3233/aic-1994-7104Bertsekas, D. P. (1988). The auction algorithm: A distributed relaxation method for the assignment problem. Annals of Operations Research, 14(1), 105-123. doi:10.1007/bf02186476Bertsekas, D. P., & Castanon, D. A. (1989). The auction algorithm for the transportation problem. Annals of Operations Research, 20(1), 67-96. doi:10.1007/bf0221692

ICAPS 2012. Proceedings of the third Workshop on the International Planning Competition

22nd International Conference on Automated Planning and Scheduling. June 25-29, 2012, Atibaia, Sao Paulo (Brazil). Proceedings of the 3rd the International Planning CompetitionThe Academic Advising Planning Domain / Joshua T. Guerin, Josiah P. Hanna, Libby Ferland, Nicholas Mattei, and Judy Goldsmith. -- Leveraging Classical Planners through Translations / Ronen I. Brafman, Guy Shani, and Ran Taig. -- Advances in BDD Search: Filtering, Partitioning, and Bidirectionally Blind / Stefan Edelkamp, Peter Kissmann, and Álvaro Torralba. -- A Multi-Agent Extension of PDDL3.1 / Daniel L. Kovacs. -- Mining IPC-2011 Results / Isabel Cenamor, Tomás de la Rosa, and Fernando Fernández. -- How Good is the Performance of the Best Portfolio in IPC-2011? / Sergio Nuñez, Daniel Borrajo, and Carlos Linares López. -- “Type Problem in Domain Description!” or, Outsiders’ Suggestions for PDDL Improvement / Robert P. Goldman and Peter KellerEn prens

Non-Cooperative Games for Self-Interested Planning Agents

Multi-Agent Planning (MAP) is a topic of growing interest that deals with the problem of automated planning in domains where multiple agents plan and act together in a shared environment. In most cases, agents in MAP are cooperative (altruistic) and work together towards a collaborative solution. However, when rational self-interested agents are involved in a MAP task, the ultimate objective is to find a joint plan that accomplishes the agents' local tasks while satisfying their private interests. Among the MAP scenarios that involve self-interested agents, non-cooperative MAP refers to problems where non-strictly competitive agents feature common and conflicting interests. In this setting, conflicts arise when self-interested agents put their plans together and the resulting combination renders some of the plans non-executable, which implies a utility loss for the affected agents. Each participant wishes to execute its plan as it was conceived, but congestion issues and conflicts among the actions of the different plans compel agents to find a coordinated stable solution. Non-cooperative MAP tasks are tackled through non-cooperative games, which aim at finding a stable (equilibrium) joint plan that ensures the agents' plans are executable (by addressing planning conflicts) while accounting for their private interests as much as possible. Although this paradigm reflects many real-life problems, there is a lack of computational approaches to non-cooperative MAP in the literature. This PhD thesis pursues the application of non-cooperative games to solve non-cooperative MAP tasks that feature rational self-interested agents. Each agent calculates a plan that attains its individual planning task, and subsequently, the participants try to execute their plans in a shared environment. We tackle non-cooperative MAP from a twofold perspective. On the one hand, we focus on agents' satisfaction by studying desirable properties of stable solutions, such as optimality and fairness. On the other hand, we look for a combination of MAP and game-theoretic techniques capable of efficiently computing stable joint plans while minimizing the computational complexity of this combined task. Additionally, we consider planning conflicts and congestion issues in the agents' utility functions, which results in a more realistic approach. To the best of our knowledge, this PhD thesis opens up a new research line in non-cooperative MAP and establishes the basic principles to attain the problem of synthesizing stable joint plans for self-interested planning agents through the combination of game theory and automated planning.La Planificación Multi-Agente (PMA) es un tema de creciente interés que trata el problema de la planificación automática en dominios donde múltiples agentes planifican y actúan en un entorno compartido. En la mayoría de casos, los agentes en PMA son cooperativos (altruistas) y trabajan juntos para obtener una solución colaborativa. Sin embargo, cuando los agentes involucrados en una tarea de PMA son racionales y auto-interesados, el objetivo último es obtener un plan conjunto que resuelva las tareas locales de los agentes y satisfaga sus intereses privados. De entre los distintos escenarios de PMA que involucran agentes auto-interesados, la PMA no cooperativa se centra en problemas que presentan un conjunto de agentes no estrictamente competitivos con intereses comunes y conflictivos. En este contexto, pueden surgir conflictos cuando los agentes ponen en común sus planes y la combinación resultante provoca que algunos de estos planes no sean ejecutables, lo que implica una pérdida de utilidad para los agentes afectados. Cada participante desea ejecutar su plan tal como fue concebido, pero las congestiones y conflictos que pueden surgir entre las acciones de los diferentes planes fuerzan a los agentes a obtener una solución estable y coordinada. Las tareas de PMA no cooperativa se abordan a través de juegos no cooperativos, cuyo objetivo es hallar un plan conjunto estable (equilibrio) que asegure que los planes de los agentes sean ejecutables (resolviendo los conflictos de planificación) al tiempo que los agentes satisfacen sus intereses privados en la medida de lo posible. Aunque este paradigma refleja muchos problemas de la vida real, existen pocos enfoques computacionales para PMA no cooperativa en la literatura. Esta tesis doctoral estudia el uso de juegos no cooperativos para resolver tareas de PMA no cooperativa con agentes racionales auto-interesados. Cada agente calcula un plan para su tarea de planificación y posteriormente, los participantes intentan ejecutar sus planes en un entorno compartido. Abordamos la PMA no cooperativa desde una doble perspectiva. Por una parte, nos centramos en la satisfacción de los agentes estudiando las propiedades deseables de soluciones estables, tales como la optimalidad y la justicia. Por otra parte, buscamos una combinación de PMA y técnicas de teoría de juegos capaz de calcular planes conjuntos estables de forma eficiente al tiempo que se minimiza la complejidad computacional de esta tarea combinada. Además, consideramos los conflictos de planificación y congestiones en las funciones de utilidad de los agentes, lo que resulta en un enfoque más realista. Bajo nuestro punto de vista, esta tesis doctoral abre una nueva línea de investigación en PMA no cooperativa y establece los principios básicos para resolver el problema de la generación de planes conjuntos estables para agentes de planificación auto-interesados mediante la combinación de teoría de juegos y planificación automática.La Planificació Multi-Agent (PMA) és un tema de creixent interès que tracta el problema de la planificació automàtica en dominis on múltiples agents planifiquen i actuen en un entorn compartit. En la majoria de casos, els agents en PMA són cooperatius (altruistes) i treballen junts per obtenir una solució col·laborativa. No obstant això, quan els agents involucrats en una tasca de PMA són racionals i auto-interessats, l'objectiu últim és obtenir un pla conjunt que resolgui les tasques locals dels agents i satisfaci els seus interessos privats. D'entre els diferents escenaris de PMA que involucren agents auto-interessats, la PMA no cooperativa se centra en problemes que presenten un conjunt d'agents no estrictament competitius amb interessos comuns i conflictius. En aquest context, poden sorgir conflictes quan els agents posen en comú els seus plans i la combinació resultant provoca que alguns d'aquests plans no siguin executables, el que implica una pèrdua d'utilitat per als agents afectats. Cada participant vol executar el seu pla tal com va ser concebut, però les congestions i conflictes que poden sorgir entre les accions dels diferents plans forcen els agents a obtenir una solució estable i coordinada. Les tasques de PMA no cooperativa s'aborden a través de jocs no cooperatius, en els quals l'objectiu és trobar un pla conjunt estable (equilibri) que asseguri que els plans dels agents siguin executables (resolent els conflictes de planificació) alhora que els agents satisfan els seus interessos privats en la mesura del possible. Encara que aquest paradigma reflecteix molts problemes de la vida real, hi ha pocs enfocaments computacionals per PMA no cooperativa en la literatura. Aquesta tesi doctoral estudia l'ús de jocs no cooperatius per resoldre tasques de PMA no cooperativa amb agents racionals auto-interessats. Cada agent calcula un pla per a la seva tasca de planificació i posteriorment, els participants intenten executar els seus plans en un entorn compartit. Abordem la PMA no cooperativa des d'una doble perspectiva. D'una banda, ens centrem en la satisfacció dels agents estudiant les propietats desitjables de solucions estables, com ara la optimalitat i la justícia. D'altra banda, busquem una combinació de PMA i tècniques de teoria de jocs capaç de calcular plans conjunts estables de forma eficient alhora que es minimitza la complexitat computacional d'aquesta tasca combinada. A més, considerem els conflictes de planificació i congestions en les funcions d'utilitat dels agents, el que resulta en un enfocament més realista. Des del nostre punt de vista, aquesta tesi doctoral obre una nova línia d'investigació en PMA no cooperativa i estableix els principis bàsics per resoldre el problema de la generació de plans conjunts estables per a agents de planificació auto-interessats mitjançant la combinació de teoria de jocs i planificació automàtica.Jordán Prunera, JM. (2017). Non-Cooperative Games for Self-Interested Planning Agents [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/90417TESI

Managing time budgets shared between planning and execution

Agents operating in domains with time budgets shared between planning and execution must carefully balance the need to plan versus the need to act. This is because planning and execution consume the same time resource. Excessive planning can delay the time it takes to achieve a goal, and so reduce the reward attained by an agent. Whereas, insufficient planning will mean the agent creates and executes low reward plans. This thesis looks at three ways to increase the reward achieved by an agent in domains with shared time budgets. The first way is by optimising time allocated to planning, using two different methods -- an optimal plan duration predictor and an online loss limiter. A second is by finding ways to act in a goal-directed manner during planning. We look at using previous plans or new plans generated quickly as heuristics for acting whilst planning. In addition, we present a way of describing actions that are mid-execution to speed the transition between planning and execution. Lastly, this thesis presents a way in which to manage time budgets in multi-agent domains. We use market-based task allocation with deadlines to produce faster task allocation and planning

Argumentation-based methods for multi-perspective cooperative planning

Through cooperation, agents can transcend their individual capabilities and achieve goals that would be unattainable otherwise. Existing multiagent planning work considers each agent’s action capabilities, but does not account for distributed knowledge and the incompatible views agents may have of the planning domain. These divergent views can be a result of faulty sensors, local and incomplete knowledge, and outdated information, or simply because each agent has conducted different inferences and their beliefs are not aligned. This thesis is concerned with Multi-Perspective Cooperative Planning (MPCP), the problem of synthesising a plan for multiple agents which share a goal but hold different views about the state of the environment and the specification of the actions they can perform to affect it. Reaching agreement on a mutually acceptable plan is important, since cautious autonomous agents will not subscribe to plans that they individually believe to be inappropriate or even potentially hazardous. We specify the MPCP problem by adapting standard set-theoretic planning notation. Based on argumentation theory we define a new notion of plan acceptability, and introduce a novel formalism that combines defeasible logic programming and situation calculus that enables the succinct axiomatisation of contradictory planning theories and allows deductive argumentation-based inference. Our work bridges research in argumentation, reasoning about action and classical planning. We present practical methods for reasoning and planning with MPCP problems that exploit the inherent structure of planning domains and efficient planning heuristics. Finally, in order to allow distribution of tasks, we introduce a family of argumentation-based dialogue protocols that enable the agents to reach agreement on plans in a decentralised manner. Based on the concrete foundation of deductive argumentation we analytically investigate important properties of our methods illustrating the correctness of the proposed planning mechanisms. We also empirically evaluate the efficiency of our algorithms in benchmark planning domains. Our results illustrate that our methods can synthesise acceptable plans within reasonable time in large-scale domains, while maintaining a level of expressiveness comparable to that of modern automated planning