Application of temporal difference learning and supervised learning in the game of Go.

by Horace Wai-Kit, Chan.Thesis (M.Phil.)--Chinese University of Hong Kong, 1996.Includes bibliographical references (leaves 109-112).Acknowledgement --- p.iAbstract --- p.iiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Overview --- p.1Chapter 1.2 --- Objective --- p.3Chapter 1.3 --- Organization of This Thesis --- p.3Chapter 2 --- Background --- p.5Chapter 2.1 --- Definitions --- p.5Chapter 2.1.1 --- Theoretical Definition of Solving a Game --- p.5Chapter 2.1.2 --- Definition of Computer Go --- p.7Chapter 2.2 --- State of the Art of Computer Go --- p.7Chapter 2.3 --- A Framework for Computer Go --- p.11Chapter 2.3.1 --- Evaluation Function --- p.11Chapter 2.3.2 --- Plausible Move Generator --- p.14Chapter 2.4 --- Problems Tackled in this Research --- p.14Chapter 3 --- Application of TD in Game Playing --- p.15Chapter 3.1 --- Introduction --- p.15Chapter 3.2 --- Reinforcement Learning and TD Learning --- p.15Chapter 3.2.1 --- Models of Learning --- p.16Chapter 3.2.2 --- Temporal Difference Learning --- p.16Chapter 3.3 --- TD Learning and Game-playing --- p.20Chapter 3.3.1 --- Game-Playing as a Delay-reward Prediction Problem --- p.20Chapter 3.3.2 --- Previous Work of TD Learning in Backgammon --- p.20Chapter 3.3.3 --- Previous Works of TD Learning in Go --- p.22Chapter 3.4 --- Design of this Research --- p.23Chapter 3.4.1 --- Limitations in the Previous Researches --- p.24Chapter 3.4.2 --- Motivation --- p.25Chapter 3.4.3 --- Objective and Methodology --- p.26Chapter 4 --- Deriving a New Updating Rule to Apply TD Learning in Multi-layer Perceptron --- p.28Chapter 4.1 --- Multi-layer Perceptron (MLP) --- p.28Chapter 4.2 --- Derivation of TD(A) Learning Rule for MLP --- p.31Chapter 4.2.1 --- Notations --- p.31Chapter 4.2.2 --- A New Generalized Delta Rule --- p.31Chapter 4.2.3 --- Updating rule for TD(A) Learning --- p.34Chapter 4.3 --- Algorithm of Training MLP using TD(A) --- p.35Chapter 4.3.1 --- Definitions of Variables in the Algorithm --- p.35Chapter 4.3.2 --- Training Algorithm --- p.36Chapter 4.3.3 --- Description of the Algorithm --- p.39Chapter 5 --- Experiments --- p.41Chapter 5.1 --- Introduction --- p.41Chapter 5.2 --- Experiment 1 : Training Evaluation Function for 7 x 7 Go Games by TD(λ) with Self-playing --- p.42Chapter 5.2.1 --- Introduction --- p.42Chapter 5.2.2 --- 7 x 7 Go --- p.42Chapter 5.2.3 --- Experimental Designs --- p.43Chapter 5.2.4 --- Performance Testing for Trained Networks --- p.44Chapter 5.2.5 --- Results --- p.44Chapter 5.2.6 --- Discussions --- p.45Chapter 5.2.7 --- Limitations --- p.47Chapter 5.3 --- Experiment 2 : Training Evaluation Function for 9 x 9 Go Games by TD(λ) Learning from Human Games --- p.47Chapter 5.3.1 --- Introduction --- p.47Chapter 5.3.2 --- 9x 9 Go game --- p.48Chapter 5.3.3 --- Training Data Preparation --- p.49Chapter 5.3.4 --- Experimental Designs --- p.50Chapter 5.3.5 --- Results --- p.52Chapter 5.3.6 --- Discussion --- p.54Chapter 5.3.7 --- Limitations --- p.56Chapter 5.4 --- Experiment 3 : Life Status Determination in the Go Endgame --- p.57Chapter 5.4.1 --- Introduction --- p.57Chapter 5.4.2 --- Training Data Preparation --- p.58Chapter 5.4.3 --- Experimental Designs --- p.60Chapter 5.4.4 --- Results --- p.64Chapter 5.4.5 --- Discussion --- p.65Chapter 5.4.6 --- Limitations --- p.66Chapter 5.5 --- A Postulated Model --- p.66Chapter 6 --- Conclusions --- p.69Chapter 6.1 --- Future Direction of Research --- p.71Chapter A --- An Introduction to Go --- p.72Chapter A.l --- A Brief Introduction --- p.72Chapter A.1.1 --- What is Go? --- p.72Chapter A.1.2 --- History of Go --- p.72Chapter A.1.3 --- Equipment used in a Go game --- p.73Chapter A.2 --- Basic Rules in Go --- p.74Chapter A.2.1 --- A Go game --- p.74Chapter A.2.2 --- Liberty and Capture --- p.75Chapter A.2.3 --- Ko --- p.77Chapter A.2.4 --- "Eyes, Live and Death" --- p.81Chapter A.2.5 --- Seki --- p.83Chapter A.2.6 --- Endgame and Scoring --- p.83Chapter A.2.7 --- Rank and Handicap Games --- p.85Chapter A.3 --- Strategies and Tactics in Go --- p.87Chapter A.3.1 --- Strategy vs Tactics --- p.87Chapter A.3.2 --- Open-game --- p.88Chapter A.3.3 --- Middle-game --- p.91Chapter A.3.4 --- End-game --- p.92Chapter B --- Mathematical Model of Connectivity --- p.94Chapter B.1 --- Introduction --- p.94Chapter B.2 --- Basic Definitions --- p.94Chapter B.3 --- Adjacency and Connectivity --- p.96Chapter B.4 --- String and Link --- p.98Chapter B.4.1 --- String --- p.98Chapter B.4.2 --- Link --- p.98Chapter B.5 --- Liberty and Atari --- p.99Chapter B.5.1 --- Liberty --- p.99Chapter B.5.2 --- Atari --- p.101Chapter B.6 --- Ko --- p.101Chapter B.7 --- Prohibited Move --- p.104Chapter B.8 --- Path and Distance --- p.105Bibliography --- p.10

Escape room - innovative method to teach concepts related to the periodic table

Game-based learning [1,2] can be introduced in schools to support the learning experience, thus helping students to connect the previous knowledge and the one learned from the game. Games are fun, engaging and motivating, and if they incorporate chemistry content, they could be a powerful pedagogical tool with a great educational value. Well-designed educational games develop creative thinking, inquiring and problem-solving skills, higher-level thinking skills, collaborative or cooperative learning, self-confidence and decision making [3]. They encourage discussion among students, but also with the teacher, which is a crucial part in developing thinking, clarification and correction of potential misunderstandings and misconceptions. The escape room method [4,5] has become a very popular innovative method to the chemistry teaching. Motivated by the International Year of Periodic Table [6], we created and performed several escape rooms related to the Periodic Table concepts in North Macedonia and Serbia [7,8]. Namely, we introduced educational games and puzzles among chemistry teachers within the programme for their continuing professional development in two countries. The escape room included five puzzles: Coded Message, Hidden Words, Improvised Chemistry Competition, Cool Chemistry Coffee Receipt and The Queen and the King puzzle. This is rather novice and innovative teaching practice in North Macedonia and Serbia and we are optimistic to disseminate this idea further together with the teachers in our countries. These educational games and escape rooms have a potential to increase the students’ interest and motivation to learn chemistry. References: 1. Burguillo, J. C. (2010). Using game-theory and competition-based learning to stimulate student motivation and performance. Computers & Education, 55(2), 566−575. doi: 10.1016/ j.compedu.2010.02.018 2. Pivec, M., & Dziabenko, O. (2004). Game-based learning in universities and lifelong learning: “UniGame: social skills and knowlwdgw training” game concept. Journal of Universal Computer Science, 10(1), 14−26. 3. Sung, H., & Hwang, G. (2013). A collaborative game-based learning approach to improving students’ learning performance in science courses. Computers & Education, 63, 43−51. doi: 10.1016/j.compedu.2012.11.019 4. Dietrich, N. (2018). Escape Classroom: The Leblanc Process – An Educational “Escape Game”. Journal of Chemical Education, 95, 996−999. doi: 10.1021/acs.jchemed.7b00690 5. Peleg, R., Yayon M., Katchevich D., Moria-Shipony M. & Blonder R. (2019). A Lab-Based Chemical Escape Room: Educational, Mobile, and Fun! Journal of Chemical Education, in publication. doi: 10.1021/acs.jchemed.8b00406 6. International Year of Periodic Table, https://www.iypt2019.org/ 7. Seminars for chemistry teachers (2019), Society of Chemists and Technologists of Macedonia, http://sctm.mk/seminari.htm 8. Seminars for chemistry teachers (2019), Serbian Chemical Society, https://www.shd.org.rs/index.ph

Dynamic capacity provision for wireless sensors connectivity: A profit optimization approach

[EN] We model a wireless sensors' connectivity scenario mathematically and analyze it using capacity provision mechanisms, with the objective of maximizing the profits of a network operator. The scenario has several sensors' clusters with each one having one sink node, which uploads the sensing data gathered in the cluster through the wireless connectivity of a network operator. The scenario is analyzed both as a static game and as a dynamic game, each one with two stages, using game theory. The sinks' behavior is characterized with a utility function related to the mean service time and the price paid to the operator for the service. The objective of the operator is to maximize its profits by optimizing the network capacity. In the static game, the sinks' subscription decision is modeled using a population game. In the dynamic game, the sinks' behavior is modeled using an evolutionary game and the replicator dynamic, while the operator optimal capacity is obtained solving an optimal control problem. The scenario is shown feasible from an economic point of view. In addition, the dynamic capacity provision optimization is shown as a valid mechanism for maximizing the operator profits, as well as a useful tool to analyze evolving scenarios. Finally, the dynamic analysis opens the possibility to study more complex scenarios using the differential game extension.The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Spanish Ministry of Economy and Competitiveness through project TIN2013-47272-C2-1-R; AEI/FEDER, UE through project TEC2017-85830-C2-1-P; and co-supported by the European Social Fund BES-2014-068998.Sanchis-Cano, Á.; Guijarro, L.; Condoluci, M. (2018). Dynamic capacity provision for wireless sensors connectivity: A profit optimization approach. International Journal of Distributed Sensor Networks (Online). 14(4):1-14. https://doi.org/10.1177/1550147718772544S114144Weiser, M. (1991). The Computer for the 21st Century. Scientific American, 265(3), 94-104. doi:10.1038/scientificamerican0991-94Gubbi, J., Buyya, R., Marusic, S., & Palaniswami, M. (2013). Internet of Things (IoT): A vision, architectural elements, and future directions. Future Generation Computer Systems, 29(7), 1645-1660. doi:10.1016/j.future.2013.01.010Perera, C., Zaslavsky, A., Christen, P., & Georgakopoulos, D. (2013). Sensing as a service model for smart cities supported by Internet of Things. Transactions on Emerging Telecommunications Technologies, 25(1), 81-93. doi:10.1002/ett.2704Wang, N., Hossain, E., & Bhargava, V. K. (2016). Joint Downlink Cell Association and Bandwidth Allocation for Wireless Backhauling in Two-Tier HetNets With Large-Scale Antenna Arrays. IEEE Transactions on Wireless Communications, 15(5), 3251-3268. doi:10.1109/twc.2016.2519401Chowdhury, M. Z., Jang, Y. M., & Haas, Z. J. (2013). Call admission control based on adaptive bandwidth allocation for wireless networks. Journal of Communications and Networks, 15(1), 15-24. doi:10.1109/jcn.2013.000005Nan, G., Mao, Z., Yu, M., Li, M., Wang, H., & Zhang, Y. (2014). Stackelberg Game for Bandwidth Allocation in Cloud-Based Wireless Live-Streaming Social Networks. IEEE Systems Journal, 8(1), 256-267. doi:10.1109/jsyst.2013.2253420Zhu, K., Niyato, D., Wang, P., & Han, Z. (2012). Dynamic Spectrum Leasing and Service Selection in Spectrum Secondary Market of Cognitive Radio Networks. IEEE Transactions on Wireless Communications, 11(3), 1136-1145. doi:10.1109/twc.2012.010312.110732Vamvakas, P., Tsiropoulou, E. E., & Papavassiliou, S. (2017). Dynamic Provider Selection & Power Resource Management in Competitive Wireless Communication Markets. Mobile Networks and Applications, 23(1), 86-99. doi:10.1007/s11036-017-0885-yNiyato, D., Hoang, D. T., Luong, N. C., Wang, P., Kim, D. I., & Han, Z. (2016). Smart data pricing models for the internet of things: a bundling strategy approach. IEEE Network, 30(2), 18-25. doi:10.1109/mnet.2016.7437020Guijarro, L., Pla, V., Vidal, J. R., & Naldi, M. (2016). Maximum-Profit Two-Sided Pricing in Service Platforms Based on Wireless Sensor Networks. IEEE Wireless Communications Letters, 5(1), 8-11. doi:10.1109/lwc.2015.2487259Romero, J., Guijarro, L., Pla, V., & Vidal, J. R. (2017). Price competition between a macrocell and a small-cell service provider with limited resources and optimal bandwidth user subscription: a game-theoretical model. Telecommunication Systems, 67(2), 195-209. doi:10.1007/s11235-017-0331-2Al Daoud, A., Alanyali, M., & Starobinski, D. (2010). Pricing Strategies for Spectrum Lease in Secondary Markets. IEEE/ACM Transactions on Networking, 18(2), 462-475. doi:10.1109/tnet.2009.2031176Do, C. T., Tran, N. H., Huh, E.-N., Hong, C. S., Niyato, D., & Han, Z. (2016). Dynamics of service selection and provider pricing game in heterogeneous cloud market. Journal of Network and Computer Applications, 69, 152-165. doi:10.1016/j.jnca.2016.04.012Tsiropoulou, E. E., Vamvakas, P., & Papavassiliou, S. (2017). Joint Customized Price and Power Control for Energy-Efficient Multi-Service Wireless Networks via S-Modular Theory. IEEE Transactions on Green Communications and Networking, 1(1), 17-28. doi:10.1109/tgcn.2017.2678207Sanchis-Cano, A., Romero, J., Sacoto-Cabrera, E., & Guijarro, L. (2017). Economic Feasibility of Wireless Sensor Network-Based Service Provision in a Duopoly Setting with a Monopolist Operator. Sensors, 17(12), 2727. doi:10.3390/s17122727Weber, T. A. (2011). Optimal Control Theory with Applications in Economics. doi:10.7551/mitpress/9780262015738.001.0001Mandjes, M. (2003). Pricing strategies under heterogeneous service requirements. Computer Networks, 42(2), 231-249. doi:10.1016/s1389-1286(03)00191-9Shariatmadari, H., Ratasuk, R., Iraji, S., Laya, A., Taleb, T., Jäntti, R., & Ghosh, A. (2015). Machine-type communications: current status and future perspectives toward 5G systems. IEEE Communications Magazine, 53(9), 10-17. doi:10.1109/mcom.2015.7263367Ng, C.-H., & Soong, B.-H. (2008). Queueing Modelling Fundamentals. doi:10.1002/9780470994672Mendelson, H. (1985). Pricing computer services: queueing effects. Communications of the ACM, 28(3), 312-321. doi:10.1145/3166.3171Altman, E., Boulogne, T., El-Azouzi, R., Jiménez, T., & Wynter, L. (2006). A survey on networking games in telecommunications. Computers & Operations Research, 33(2), 286-311. doi:10.1016/j.cor.2004.06.005Belleflamme, P., & Peitz, M. (2015). Industrial Organization. doi:10.1017/cbo9781107707139Reynolds, S. S. (1987). Capacity Investment, Preemption and Commitment in an Infinite Horizon Model. International Economic Review, 28(1), 69. doi:10.2307/2526860Barron, E. N. (2013). Game Theory. doi:10.1002/9781118547168Sandholm, W. (2009). Pairwise Comparison Dynamics and Evolutionary Foundations for Nash Equilibrium. Games, 1(1), 3-17. doi:10.3390/g1010003Schlag, K. H. (1998). Why Imitate, and If So, How? Journal of Economic Theory, 78(1), 130-156. doi:10.1006/jeth.1997.234

Agent-based models of competing population.

Yip Kin Fung.Thesis (M.Phil.)--Chinese University of Hong Kong, 2003.Includes bibliographical references (leaves 101-104).Abstracts in English and Chinese.Chapter 1 --- Introduction --- p.1Chapter 2 --- The Distribution of Fluctuations in Financial Data --- p.5Chapter 2.1 --- Empirical Statistics --- p.5Chapter 2.2 --- Data analyzed --- p.8Chapter 2.3 --- Levy Distribution --- p.10Chapter 2.4 --- Returns Distribution and Scaling Properties --- p.12Chapter 2.5 --- Volatility Clustering --- p.19Chapter 2.6 --- Conclusion --- p.21Chapter 3 --- Models of Herd behaviour in Financial Markets --- p.22Chapter 3.1 --- Cont and Bouchaud's model --- p.22Chapter 3.2 --- The Model of Egiuluz and Zimmerman --- p.24Chapter 3.3 --- EZ Model with Size-Dependent Dissociation Rates --- p.28Chapter 3.4 --- Democratic and Dictatorship Self-Organized Model --- p.31Chapter 3.5 --- Effect of Size-Dependent Fragmentation and Coagulation Prob- abilities --- p.33Chapter 3.6 --- Extensions of EZ model --- p.35Chapter 3.7 --- Conclusion --- p.39Chapter 4 --- Review on the Minority Game(MG) --- p.42Chapter 4.1 --- The Model and Results --- p.42Chapter 4.2 --- Crowd-anticrowd Theory and Phase Transition --- p.46Chapter 4.3 --- Market Efficiency --- p.48Chapter 5 --- MG with biased strategy pool --- p.52Chapter 5.1 --- The Model --- p.53Chapter 5.2 --- Numerical Results and Discussion --- p.53Chapter 5.3 --- Theory: MG with Biased Strategy Pool --- p.61Chapter 5.4 --- Conclusion --- p.69Chapter 6 --- MG with Randomly Participating Agents --- p.71Chapter 6.1 --- The Model with One RPA --- p.71Chapter 6.2 --- Results for q = 0.5 --- p.72Chapter 6.3 --- Inefficiency and Success Rate --- p.76Chapter 6.4 --- Results for q ≠ 0.5 --- p.82Chapter 6.5 --- Many RPAs --- p.85Chapter 6.6 --- Conclusion --- p.86Chapter 7 --- A Model on Coupled Minority Games --- p.88Chapter 7.1 --- The Model --- p.89Chapter 7.2 --- Results and Discussion。 --- p.90Chapter 7.3 --- Conclusion --- p.95Chapter 8 --- Conclusion --- p.97Bibliography --- p.101Chapter A --- Solving Cluster Size distribution in EZ model --- p.10

An Abstract Framework for Non-Cooperative Multi-Agent Planning

[EN] In non-cooperative multi-agent planning environments, it is essential to have a system that enables the agents¿ strategic behavior. It is also important to consider all planning phases, i.e., goal allocation, strategic planning, and plan execution, in order to solve a complete problem. Currently, we have no evidence of the existence of any framework that brings together all these phases for non-cooperative multi-agent planning environments. In this work, an exhaustive study is made to identify existing approaches for the different phases as well as frameworks and different applicable techniques in each phase. Thus, an abstract framework that covers all the necessary phases to solve these types of problems is proposed. In addition, we provide a concrete instantiation of the abstract framework using different techniques to promote all the advantages that the framework can offer. A case study is also carried out to show an illustrative example of how to solve a non-cooperative multi-agent planning problem with the presented framework. This work aims to establish a base on which to implement all the necessary phases using the appropriate technologies in each of them and to solve complex problems in different domains of application for non-cooperative multi-agent planning settings.This work was partially funded by MINECO/FEDER RTI2018-095390-B-C31 project of the Spanish government. Jaume Jordan and Vicent Botti are funded by Universitat Politecnica de Valencia (UPV) PAID-06-18 project. Jaume Jordan is also funded by grant APOSTD/2018/010 of Generalitat Valenciana Fondo Social Europeo.Jordán, J.; Bajo, J.; Botti, V.; Julian Inglada, VJ. (2019). An Abstract Framework for Non-Cooperative Multi-Agent Planning. Applied Sciences. 9(23):1-18. https://doi.org/10.3390/app9235180S118923De Weerdt, M., & Clement, B. (2009). Introduction to planning in multiagent systems. Multiagent and Grid Systems, 5(4), 345-355. doi:10.3233/mgs-2009-0133Dunne, P. E., Kraus, S., Manisterski, E., & Wooldridge, M. (2010). Solving coalitional resource games. Artificial Intelligence, 174(1), 20-50. doi:10.1016/j.artint.2009.09.005Torreño, A., Onaindia, E., Komenda, A., & Štolba, M. (2018). Cooperative Multi-Agent Planning. ACM Computing Surveys, 50(6), 1-32. doi:10.1145/3128584Fikes, R. E., & Nilsson, N. J. (1971). Strips: A new approach to the application of theorem proving to problem solving. Artificial Intelligence, 2(3-4), 189-208. doi:10.1016/0004-3702(71)90010-5Hoffmann, J., & Nebel, B. (2001). The FF Planning System: Fast Plan Generation Through Heuristic Search. Journal of Artificial Intelligence Research, 14, 253-302. doi:10.1613/jair.855Dukeman, A., & Adams, J. A. (2017). Hybrid mission planning with coalition formation. Autonomous Agents and Multi-Agent Systems, 31(6), 1424-1466. doi:10.1007/s10458-017-9367-7Hadad, M., Kraus, S., Ben-Arroyo Hartman, I., & Rosenfeld, A. (2013). Group planning with time constraints. Annals of Mathematics and Artificial Intelligence, 69(3), 243-291. doi:10.1007/s10472-013-9363-9Guo, Y., Pan, Q., Sun, Q., Zhao, C., Wang, D., & Feng, M. (2019). Cooperative Game-based Multi-Agent Path Planning with Obstacle Avoidance*. 2019 IEEE 28th International Symposium on Industrial Electronics (ISIE). doi:10.1109/isie.2019.8781205v. Neumann, J. (1928). Zur Theorie der Gesellschaftsspiele. Mathematische Annalen, 100(1), 295-320. doi:10.1007/bf01448847Mookherjee, D., & Sopher, B. (1994). Learning Behavior in an Experimental Matching Pennies Game. Games and Economic Behavior, 7(1), 62-91. doi:10.1006/game.1994.1037Ochs, J. (1995). Games with Unique, Mixed Strategy Equilibria: An Experimental Study. Games and Economic Behavior, 10(1), 202-217. doi:10.1006/game.1995.1030Applegate, C., Elsaesser, C., & Sanborn, J. (1990). An architecture for adversarial planning. IEEE Transactions on Systems, Man, and Cybernetics, 20(1), 186-194. doi:10.1109/21.47820Sailer, F., Buro, M., & Lanctot, M. (2007). Adversarial Planning Through Strategy Simulation. 2007 IEEE Symposium on Computational Intelligence and Games. doi:10.1109/cig.2007.368082Willmott, S., Richardson, J., Bundy, A., & Levine, J. (2001). Applying adversarial planning techniques to Go. Theoretical Computer Science, 252(1-2), 45-82. doi:10.1016/s0304-3975(00)00076-1Nau, D. S., Au, T. C., Ilghami, O., Kuter, U., Murdock, J. W., Wu, D., & Yaman, F. (2003). SHOP2: An HTN Planning System. Journal of Artificial Intelligence Research, 20, 379-404. doi:10.1613/jair.1141Knuth, D. E., & Moore, R. W. (1975). An analysis of alpha-beta pruning. Artificial Intelligence, 6(4), 293-326. doi:10.1016/0004-3702(75)90019-3Vickrey, W. (1961). COUNTERSPECULATION, AUCTIONS, AND COMPETITIVE SEALED TENDERS. The Journal of Finance, 16(1), 8-37. doi:10.1111/j.1540-6261.1961.tb02789.xClarke, E. H. (1971). Multipart pricing of public goods. Public Choice, 11(1), 17-33. doi:10.1007/bf01726210Groves, T. (1973). Incentives in Teams. Econometrica, 41(4), 617. doi:10.2307/1914085Savaux, J., Vion, J., Piechowiak, S., Mandiau, R., Matsui, T., Hirayama, K., … Silaghi, M. (2016). DisCSPs with Privacy Recast as Planning Problems for Self-Interested Agents. 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI). doi:10.1109/wi.2016.0057Buzing, P., Mors, A. ter, Valk, J., & Witteveen, C. (2006). Coordinating Self-interested Planning Agents. Autonomous Agents and Multi-Agent Systems, 12(2), 199-218. doi:10.1007/s10458-005-6104-4Ter Mors, A., & Witteveen, C. (s. f.). Coordinating Non Cooperative Planning Agents: Complexity Results. IEEE/WIC/ACM International Conference on Intelligent Agent Technology. doi:10.1109/iat.2005.60Hrnčíř, J., Rovatsos, M., & Jakob, M. (2015). Ridesharing on Timetabled Transport Services: A Multiagent Planning Approach. Journal of Intelligent Transportation Systems, 19(1), 89-105. doi:10.1080/15472450.2014.941759Galuszka, A., & Swierniak, A. (2009). Planning in Multi-agent Environment Using Strips Representation and Non-cooperative Equilibrium Strategy. Journal of Intelligent and Robotic Systems, 58(3-4), 239-251. doi:10.1007/s10846-009-9364-4Rosenthal, R. W. (1973). A class of games possessing pure-strategy Nash equilibria. International Journal of Game Theory, 2(1), 65-67. doi:10.1007/bf01737559Jordán, J., Torreño, A., de Weerdt, M., & Onaindia, E. (2017). A better-response strategy for self-interested planning agents. Applied Intelligence, 48(4), 1020-1040. doi:10.1007/s10489-017-1046-5Veloso, M., Muñoz-Avila, H., & Bergmann, R. (1996). Case-based planning: selected methods and systems. AI Communications, 9(3), 128-137. doi:10.3233/aic-1996-9305VOORNEVELD, M., BORM, P., VAN MEGEN, F., TIJS, S., & FACCHINI, G. (1999). CONGESTION GAMES AND POTENTIALS RECONSIDERED. International Game Theory Review, 01(03n04), 283-299. doi:10.1142/s0219198999000219Han-Lim Choi, Brunet, L., & How, J. P. (2009). Consensus-Based Decentralized Auctions for Robust Task Allocation. IEEE Transactions on Robotics, 25(4), 912-926. doi:10.1109/tro.2009.2022423Monderer, D., & Shapley, L. S. (1996). Potential Games. Games and Economic Behavior, 14(1), 124-143. doi:10.1006/game.1996.0044Friedman, J. W., & Mezzetti, C. (2001). Learning in Games by Random Sampling. Journal of Economic Theory, 98(1), 55-84. doi:10.1006/jeth.2000.2694Aamodt, A., & Plaza, E. (1994). Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches. AI Communications, 7(1), 39-59. doi:10.3233/aic-1994-7104Bertsekas, D. P. (1988). The auction algorithm: A distributed relaxation method for the assignment problem. Annals of Operations Research, 14(1), 105-123. doi:10.1007/bf02186476Bertsekas, D. P., & Castanon, D. A. (1989). The auction algorithm for the transportation problem. Annals of Operations Research, 20(1), 67-96. doi:10.1007/bf0221692

Zielonka's Recursive Algorithm: dull, weak and solitaire games and tighter bounds

Dull, weak and nested solitaire games are important classes of parity games, capturing, among others, alternation-free mu-calculus and ECTL* model checking problems. These classes can be solved in polynomial time using dedicated algorithms. We investigate the complexity of Zielonka's Recursive algorithm for solving these special games, showing that the algorithm runs in O(d (n + m)) on weak games, and, somewhat surprisingly, that it requires exponential time to solve dull games and (nested) solitaire games. For the latter classes, we provide a family of games G, allowing us to establish a lower bound of 2^(n/3). We show that an optimisation of Zielonka's algorithm permits solving games from all three classes in polynomial time. Moreover, we show that there is a family of (non-special) games M that permits us to establish a lower bound of 2^(n/3), improving on the previous lower bound for the algorithm.Comment: In Proceedings GandALF 2013, arXiv:1307.416

A Comparison of BDD-Based Parity Game Solvers

Parity games are two player games with omega-winning conditions, played on finite graphs. Such games play an important role in verification, satisfiability and synthesis. It is therefore important to identify algorithms that can efficiently deal with large games that arise from such applications. In this paper, we describe our experiments with BDD-based implementations of four parity game solving algorithms, viz. Zielonka's recursive algorithm, the more recent Priority Promotion algorithm, the Fixpoint-Iteration algorithm and the automata based APT algorithm. We compare their performance on several types of random games and on a number of cases taken from the Keiren benchmark set.Comment: In Proceedings GandALF 2018, arXiv:1809.0241

Local Strategy Improvement for Parity Game Solving

The problem of solving a parity game is at the core of many problems in model checking, satisfiability checking and program synthesis. Some of the best algorithms for solving parity game are strategy improvement algorithms. These are global in nature since they require the entire parity game to be present at the beginning. This is a distinct disadvantage because in many applications one only needs to know which winning region a particular node belongs to, and a witnessing winning strategy may cover only a fractional part of the entire game graph. We present a local strategy improvement algorithm which explores the game graph on-the-fly whilst performing the improvement steps. We also compare it empirically with existing global strategy improvement algorithms and the currently only other local algorithm for solving parity games. It turns out that local strategy improvement can outperform these others by several orders of magnitude