PCGRL: Procedural Content Generation via Reinforcement Learning

We investigate how reinforcement learning can be used to train level-designing agents. This represents a new approach to procedural content generation in games, where level design is framed as a game, and the content generator itself is learned. By seeing the design problem as a sequential task, we can use reinforcement learning to learn how to take the next action so that the expected final level quality is maximized. This approach can be used when few or no examples exist to train from, and the trained generator is very fast. We investigate three different ways of transforming two-dimensional level design problems into Markov decision processes and apply these to three game environments.Comment: 7 pages, 7 figures, 1 table, published at AIIDE202

Amorphous Fortress: Observing Emergent Behavior in Multi-Agent FSMs

We introduce a system called Amorphous Fortress -- an abstract, yet spatial, open-ended artificial life simulation. In this environment, the agents are represented as finite-state machines (FSMs) which allow for multi-agent interaction within a constrained space. These agents are created by randomly generating and evolving the FSMs; sampling from pre-defined states and transitions. This environment was designed to explore the emergent AI behaviors found implicitly in simulation games such as Dwarf Fortress or The Sims. We apply the hill-climber evolutionary search algorithm to this environment to explore the various levels of depth and interaction from the generated FSMs.Comment: 9 pages; Accepted to the 1st ALIFE for and from video games Workshop 202

Controllable Path of Destruction

Path of Destruction (PoD) is a self-supervised method for learning iterative generators. The core idea is to produce a training set by destroying a set of artifacts, and for each destructive step create a training instance based on the corresponding repair action. A generator trained on this dataset can then generate new artifacts by ``repairing'' from arbitrary states. The PoD method is very data-efficient in terms of original training examples and well-suited to functional artifacts composed of categorical data, such as game levels and discrete 3D structures. In this paper, we extend the Path of Destruction method to allow designer control over aspects of the generated artifacts. Controllability is introduced by adding conditional inputs to the state-action pairs that make up the repair trajectories. We test the controllable PoD method in a 2D dungeon setting, as well as in the domain of small 3D Lego cars.Comment: 8 pages, 6 figures, and 2 tables. Published at CoG Conference 202

Evolutionary Machine Learning and Games

Evolutionary machine learning (EML) has been applied to games in multiple ways, and for multiple different purposes. Importantly, AI research in games is not only about playing games; it is also about generating game content, modeling players, and many other applications. Many of these applications pose interesting problems for EML. We will structure this chapter on EML for games based on whether evolution is used to augment machine learning (ML) or ML is used to augment evolution. For completeness, we also briefly discuss the usage of ML and evolution separately in games.Comment: 27 pages, 5 figures, part of Evolutionary Machine Learning Book (https://link.springer.com/book/10.1007/978-981-99-3814-8

LLMatic: Neural Architecture Search via Large Language Models and Quality-Diversity Optimization

Large Language Models (LLMs) have emerged as powerful tools capable of accomplishing a broad spectrum of tasks. Their abilities span numerous areas, and one area where they have made a significant impact is in the domain of code generation. In this context, we view LLMs as mutation and crossover tools. Meanwhile, Quality-Diversity (QD) algorithms are known to discover diverse and robust solutions. By merging the code-generating abilities of LLMs with the diversity and robustness of QD solutions, we introduce LLMatic, a Neural Architecture Search (NAS) algorithm. While LLMs struggle to conduct NAS directly through prompts, LLMatic uses a procedural approach, leveraging QD for prompts and network architecture to create diverse and highly performant networks. We test LLMatic on the CIFAR-10 image classification benchmark, demonstrating that it can produce competitive networks with just $2,000$ searches, even without prior knowledge of the benchmark domain or exposure to any previous top-performing models for the benchmark

Level Generation Through Large Language Models

Large Language Models (LLMs) are powerful tools, capable of leveraging their training on natural language to write stories, generate code, and answer questions. But can they generate functional video game levels? Game levels, with their complex functional constraints and spatial relationships in more than one dimension, are very different from the kinds of data an LLM typically sees during training. Datasets of game levels are also hard to come by, potentially taxing the abilities of these data-hungry models. We investigate the use of LLMs to generate levels for the game Sokoban, finding that LLMs are indeed capable of doing so, and that their performance scales dramatically with dataset size. We also perform preliminary experiments on controlling LLM level generators and discuss promising areas for future work

Spinal Cord Stimulation for Neuropathic Pain in England From 2010 to 2020: A Hospital Episode Statistics Analysis.

Spinal cord stimulation (SCS) is a recognized intervention for the management of chronic neuropathic pain. The United Kingdom National Institute of Health and Care Excellence has recommended SCS as a management option for chronic neuropathic pain since 2008. The aim of this study is to undertake an assessment of SCS uptake across the National Health Service in England up to 2020. Hospital Episode Statistics were obtained for patients with neuropathic pain potentially eligible for SCS and patients receiving an SCS-related procedure. Data were retrieved nationally and per region from the years 2010-2011 to 2019-2020. There were 50,288 adults in England attending secondary care with neuropathic pain in 2010-2011, increasing to 66,376 in 2019-2020. The number of patients with neuropathic pain with an SCS procedure increased on a year-to-year basis until 2018-2019. However, less than 1% of people with neuropathic pain received an SCS device with no evidence of an increase over time when considering the background increase in neuropathic pain prevalence. Only a small proportion of patients in England with neuropathic pain potentially eligible for SCS receives this intervention. The recommendation for routine use of SCS for management of neuropathic pain has not resulted in an uptake of SCS over the last decade

Factors affecting the survival of harbor ( Phoca vitulina ) and gray seal ( Halichoerus grypus ) juveniles admitted for rehabilitation in the UK and Ireland

From Wiley via Jisc Publications RouterHistory: received 2021-06-15, accepted 2022-09-13, pub-electronic 2022-10-14Article version: VoRPublication status: PublishedAbstract: The UK shores are home to approximately 40% of the world's population of gray seals (Halichoerus grypus) and 40% of Europe's harbor seals (Phoca vitulina). Stranded juvenile seals of both species are frequently rescued and admitted for rehabilitation. This study investigates the causes of P. vitulina and H. grypus admittance to rehabilitation centers in the UK and Ireland and identifies factors that can affect juvenile seal survival. Rehabilitation records for 1,435 P. vitulina and 2,691 H. grypus were used from five rehabilitation centers from 1988 through 2020. The most common nonexclusive reasons for seal admission to rehabilitation centers included malnourishment (37%), injuries (37%), maternal abandonment (15%), lethargy (12%), and parasite infections (8%). A mixed effects logistic regression model showed that H. grypus had 4.55 times higher survival odds than P. vitulina and that the odds of survival to release multiplied by 1.07 for every kilogram over their age‐predicted weight. This weight‐dependent survival could be attributed to the importance of fat in thermoregulation, hydration, and buoyancy during foraging. We recommend that seal rehabilitators pay special attention to the weight of admitted juvenile seals during triage and treatment to enhance their odds of survival and consequent release to the wild

Factors affecting the survival of harbor (Phoca vitulina) and gray seal (Halichoerus grypus) juveniles admitted for rehabilitation in the UK and Ireland

The UK shores are home to approximately 40% of the world's population of gray seals (Halichoerus grypus) and 40% of Europe's harbor seals (Phoca vitulina). Stranded juvenile seals of both species are frequently rescued and admitted for rehabilitation. This study investigates the causes of P. vitulina and H. grypus admittance to rehabilitation centers in the UK and Ireland and identifies factors that can affect juvenile seal survival. Rehabilitation records for 1,435 P. vitulina and 2,691 H. grypus were used from five rehabilitation centers from 1988 through 2020. The most common nonexclusive reasons for seal admission to rehabilitation centers included malnourishment (37%), injuries (37%), maternal abandonment (15%), lethargy (12%), and parasite infections (8%). A mixed effects logistic regression model showed that H. grypus had 4.55 times higher survival odds than P. vitulina and that the odds of survival to release multiplied by 1.07 for every kilogram over their age-predicted weight. This weight-dependent survival could be attributed to the importance of fat in thermoregulation, hydration, and buoyancy during foraging. We recommend that seal rehabilitators pay special attention to the weight of admitted juvenile seals during triage and treatment to enhance their odds of survival and consequent release to the wild