Monte Carlo Tree Search Applied to a Modified Pursuit/Evasion Scotland Yard Game with Rendezvous Spaceflight Operation Applications

This thesis takes the Scotland Yard board game and modifies its rules to mimic important aspects of space in order to facilitate the creation of artificial intelligence for space asset pursuit/evasion scenarios. Space has become a physical warfighting domain. To combat threats, an understanding of the tactics, techniques, and procedures must be captured and studied. Games and simulations are effective tools to capture data lacking historical context. Artificial intelligence and machine learning models can use simulations to develop proper defensive and offensive tactics, techniques, and procedures capable of protecting systems against potential threats. Monte Carlo Tree Search is a bandit-based reinforcement learning model known for using limited domain knowledge to push favorable results. Monte Carlo agents have been used in a multitude of imperfect domain knowledge games. One such game was in which Monte Carlo agents were produced and studied in an imperfect domain game for pursuit-evasion tactics is Scotland Yard. This thesis continues the Monte Carlo agents previously produced by Mark Winands and Pim Nijssen and applied to Scotland Yard. In the research presented here, the rules for Scotland Yard are analyzed and presented in an expansion that partially accounts for spaceflight dynamics in order to study the agents within a simplified model, while having some foundation for use within space environments. Results show promise for the use of Monte- Carlo agents in pursuit/evasion autonomous space scenarios while also illuminating some major challenges for future work in more realistic three-dimensional space environments

Monte Carlo Tree Search Algorithmen für das Brettspiel ”Scotland Yard”

Monte Carlo Algorithmen haben in letzter Zeit immer mehr an Bedeutung gewonnen. Vor allem das Einsetzen von Monte Carlo Algorithmen zum Erstellen und randomisierten Absuchen eines Suchbaums hat neue Wege im Bereich der Künstlichen Intelligenz geschaffen. In der vorangegangenen Arbeit von Minorics wurden für das Brettspiel Scotland Yard KIs für die Steuerung von Mister X entwickelt. Diese KI-Algorithmen haben jedoch keine Planung der Züge im klassischen Sinn vorgenommen. Eine Steuerung der Detektive wurde zudem nicht implementiert. Diese Arbeit erweitert die Ergebnisse der vorangegangenen Arbeit durch das Umsetzen von KIs zur Steuerung der Detektive und durch das Einsetzen von Monte-Carlo-Tree-Search-Algorithmen für die Zugplanung. Neben der Implementierung der einzelnen KIs steht auch deren ausführliche Evaluation im Mittelpunkt der Arbeit. Diese wurde anhand von umfassenden Testspielen durchgeführt, bei den jeweils verschiedene KIs für Mister X und die Detektive gegeneinander evaluiert werden

Monte-Carlo tree search method for the board game Scotland Yard

In the thesis we learn about the field of artificial intelligence that investigates board games and their program-based solutions. We examine Monte-Carlo tree search algorithm and transfer it to well-known board game Scotland Yard, considering advices from Nijssen and Winands. We focus mainly on the third phase of the algorithm, playout, and decide to implement it in three different ways (from less to more advanced techniques). We compare these three aproaches. We compare the win rates and computation time of simple and advanced methods. We also implement the game to the purpose of automated testing. In this game, detectives play by Monte-Carlo tree search algorithm and Mister X plays in two different ways - random and advanced. We want to test all of these six combinations on a large number of games, compare the results and explain them

Monte Carlo tree search in the board game of Scotland Yard

Because of its success in the computer game of Go, Monte Carlo Tree Search is becoming a progressively popular decision making algorithm in various domains. It has proven its strengths in singleplayer and multiplayer games with perfect information, however domain specific improvements must be introduced in games with imperfect information. In thesis existing approaches to the problem of applying the tree search to the Scotland Yard board game are described and empiricaly tested. It has turned out that heuristic selection of the possible location of the hider has the most impact on seekers performance. After testing, the MCTS player with all improvements has proven itself as a competitive opponent against the human player