Layered AI architecture for team based first person shooter video games

EP/P503795/1In this thesis an architecture, similar to subsumption architectures, is presented which uses low level behaviour modules, based on combinations of machine learning techniques, to create teams of autonomous agents cooperating via shared plans for interaction. The purpose of this is to perform effective single plan execution within multiple scenarios, using a modern team based first person shooter video game as the domain and visualiser. The main focus is showing that through basic machine learning mechanisms, applied in a multi-agent setting on sparse data, plans can be executed on game levels of varying size and shape without sacrificing team goals. It is also shown how different team members can perform locally sub-optimal operations which contribute to a globally better strategy by adding exploration data to the machine learning mechanisms. This contributes to the reinforcement learning problem of exploration versus exploitation, from a multi-agent perspective

Evolution of Robotic Behaviour Using Gene Expression Programming

The main objective in automatic robot controller development is to devise mechanisms whereby robot controllers can be developed with less reliance on human developers. One such mechanism is the use of evolutionary algorithms (EAs) to automatically develop robot controllers and occasionally, robot morphology. This area of research is referred to as evolutionary robotics (ER). Through the use of evolutionary techniques such as genetic algorithms (GAs) and genetic programming (GP), ER has shown to be a promising approach through which robust robot controllers can be developed. The standard ER techniques use monolithic evolution to evolve robot behaviour: monolithic evolution involves the use of one chromosome to code for an entire target behaviour. In complex problems, monolithic evolution has been shown to suffer from bootstrap problems; that is, a lack of improvement in fitness due to randomness in the solution set [103, 105, 100, 90]. Thus, approaches to dividing the tasks, such that the main behaviours emerge from the interaction of these simple tasks with the robot environment have been devised. These techniques include the subsumption architecture in behaviour based robotics, incremental learning and more recently the layered learning approach [55, 103, 56, 105, 136, 95]. These new techniques enable ER to develop complex controllers for autonomous robot. Work presented in this thesis extends the field of evolutionary robotics by introducing Gene Expression Programming (GEP) to the ER field. GEP is a newly developed evolutionary algorithm akin to GA and GP, which has shown great promise in optimisation problems. The presented research shows through experimentation that the unique formulation of GEP genes is sufficient for robot controller representation and development. The obtained results show that GEP is a plausible technique for ER problems. Additionally, it is shown that controllers evolved using GEP algorithm are able to adapt when introduced to new environments. Further, the capabilities of GEP chromosomes to code for more than one gene have been utilised to show that GEP can be used to evolve manually sub-divided robot behaviours. Additionally, this thesis extends the GEP algorithm by proposing two new evolutionary techniques named multigenic GEP with Linker Evolution (mgGEP-LE) and multigenic GEP with a Regulator Gene (mgGEP-RG). The results obtained from the proposed algorithms show that the new techniques can be used to automatically evolve modularity in robot behaviour. This ability to automate the process of behaviour sub-division and optimisation in a modular chromosome is unique to the GEP formulations discussed, and is an important advance in the development of machines that are able to evolve stratified behavioural architectures with little human intervention

Improving control through subsumption in the EvoTanks domain

Abstract — In this paper we further explore the potential of a decentralised controller architecture that places multi-layer perceptrons within a subsumption hierarchy. Previous research exploring this approach proved successful in generating agents that could solve problems while coping with new reactive stimuli. However there were many unresolved questions that we wished to explore. In this paper we explore the use of our architecture with iterative training, increased controller modularity and conflicting goals. Results provide some interesting insights into the potential this method could have to agent designers. I