Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants

Schelling, von Neumann, and the Event that Didn’t Occur

Thomas Schelling was recognized by the Nobel Prize committee as a pioneer in the application of game theory and rational choice analysis to problems of politics and international relations. However, although he makes frequent references in his writings to this approach, his main explorations and insights depend upon and require acknowledgment of its limitations. One of his principal concerns was how a country could engage in successful deterrence. If the behavioral assumptions that commonly underpin game theory are taken seriously and applied consistently, however, nuclear adversaries are almost certain to engage in devastating conflict, as John von Neumann forcefully asserted. The history of the last half century falsified von Neumann’s prediction, and the “event that didn’t occur” formed the subject of Schelling’s Nobel lecture. The answer to the question “why?” is the central concern of this paper

Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also considered as a testbed for AI research, due to its enormous state space, hidden information, multi-agent collaboration and so on. Thanks to the annual AIIDE and CIG competitions, a growing number of bots are proposed and being continuously improved. However, a big gap still remains between the top bot and the professional human players. One vital reason is that current bots mainly rely on predefined rules to perform macro actions. These rules are not scalable and efficient enough to cope with the large but partially observed macro state space in SC. In this paper, we propose a DRL based framework to do macro action selection. Our framework combines the reinforcement learning approach Ape-X DQN with Long-Short-Term-Memory (LSTM) to improve the macro action selection in bot. We evaluate our bot, named as LastOrder, on the AIIDE 2017 StarCraft AI competition bots set. Our bot achieves overall 83% win-rate, outperforming 26 bots in total 28 entrants

Game Theory and Prescriptive Analytics for Naval Wargaming Battle Management Aids

NPS NRP Technical ReportThe Navy is taking advantage of advances in computational technologies and data analytic methods to automate and enhance tactical decisions and support warfighters in highly complex combat environments. Novel automated techniques offer opportunities to support the tactical warfighter through enhanced situational awareness, automated reasoning and problem-solving, and faster decision timelines. This study will investigate how game theory and prescriptive analytics methods can be used to develop real-time wargaming capabilities to support warfighters in their ability to explore and evaluate the possible consequences of different tactical COAs to improve tactical missions. This study will develop a conceptual design of a real-time tactical wargaming capability. This study will explore data analytic methods including game theory, prescriptive analytics, and artificial intelligence (AI) to evaluate their potential to support real-time wargaming.N2/N6 - Information WarfareThis research is supported by funding from the Naval Postgraduate School, Naval Research Program (PE 0605853N/2098). https://nps.edu/nrpChief of Naval Operations (CNO)Approved for public release. Distribution is unlimited.