Strategies for Evolving Diverse and Effective Behaviours in Pursuit Domains

Evolutionary algorithms have a tendency to overuse and exploit particular behaviours in their search for optimality, even across separate runs. The resulting set of monotonous solutions caused by this tendency is a problem in many applications. This research explores different strategies designed to encourage an interesting set of diverse behaviours while still maintaining an appreciable level of efficacy. Embodied agents are situated within an open plane and play against each other in various pursuit game scenarios. The pursuit games consist of a single predator agent and twenty prey agents, with the goal always requiring the predator to catch as many prey as possible before the time limit is reached. The predator's controller is evolved through genetic programming while the preys' controllers are hand-crafted. The fitness of a solution is first calculated in a traditional manner. Inspired by Lehman and Stanley's novelty search strategy, the fitness is then combined with the diversity of the solution to produce the final fitness score. The original fitness score is determined by the number of captured prey, and the diversity score is determined through the combination of four behaviour measurements. Among many promising results, a particular diversity-based evaluation strategy and weighting combination was found to provide solutions that exhibit an excellent balance between diversity and efficacy. The results were analyzed quantitatively and qualitatively, showing the emergence of diverse and effective behaviours

Emergent Behaviour in Game AI: A Genetic Programming and CNN-based Approach to Intelligent Agent Design

Emergent behaviour is behaviour that arises from the interactions between the individual components of a system, rather than being explicitly programmed or designed. In this work, genetic programming is used to evolve an adaptive game AI, also known as an intelligent agent, whose job is to capture up to twenty-five prey agents in a simulated pursuit environment. For a pursuit game, the fitness score tallies each prey the predator captured during a run. The fitness is then used to evaluate each agent and choose who moves forward in the evolutionary process. A problem with only choosing the best performing agents is that genetic diversity becomes lost along the way, which can result in monotonous behaviour. Diverse behaviour helps agents perform under situations of uncertainty and creates more interesting computer opponents in video games, as it encourages the agent to explore different possibilities and adapt to changing circumstances. Inspired by the works of Cowan and Pozzuoli in diversifying agent behaviour, and Chen’s work in L-system tree evaluation, a convolutional neural network is introduced to automatically model the behaviour of each agent, something previously done manually. This involves training the convolutional neural network on a large data set of behaviours exhibited by the agents, which take the form of image-based traces. The resulting model is then used to detect interesting emergent behaviour. In the first set of experiments, the convolutional neural network is trained and tested on several sets of traces, then the performance of each run is analyzed. Results show that the convolutional neural network is capable of identifying 6 emergent behaviours with 98% accuracy. The second set of experiments combine genetic programming and the convolutional neural network in order to produce unique and interesting intelligent agents, as well as target specific behaviours. Results show that the system is able to evolve more innovative and effective agents that can operate in complex environments and could be extended to perform a wide range of tasks

A Methodology to Evolve Cooperation in Pursuit Domain using Genetic Network Programming

The design of strategies to devise teamwork and cooperation among agents is a central research issue in the field of multi-agent systems (MAS). The complexity of the cooperative strategy design can rise rapidly with increasing number of agents and their behavioral sophistication. The field of cooperative multi-agent learning promises solutions to such problems by attempting to discover agent behaviors as well as suggesting new approaches by applying machine learning techniques. Due to the difficulty in specifying a priori for an effective algorithm for multiple interacting agents, and the inherent adaptability of artificially evolved agents, recently, the use of evolutionary computation as a machining learning technique and a design process has received much attention. In this thesis, we design a methodology using an evolutionary computation technique called Genetic Network Programming (GNP) to automatically evolve teamwork and cooperation among agents in the pursuit domain. Simulation results show that our proposed methodology was effective in evolving teamwork and cooperation among agents. Compared with Genetic Programming approaches, its performance is significantly superior, its computation cost is less and the learning speed is faster. We also provide some analytical results of the proposed approach

Diversifying Emergent Behaviours with Age-Layered MAP-Elites

Emergent behaviour can arise unexpectedly as a by-product of the complex interactions of an autonomous system, and with the increasing desire for such systems, emergent behaviour has become an important area of interest for AI research. One aspect of this research is in searching for a diverse set of emergent behaviours which not only provides a useful tool for finding unwanted emergent behaviour, but also in finding interesting emergent behaviour. The multi-dimensional archive of phenotypic elites (MAP-Elites) algorithm is a popular evolutionary algorithm which returns a highly diverse set of elite solutions at the end of a run. The population is separated into a grid-like feature space defined by a set of behaviour dimensions specified by the user where each cell of the grid corresponds to a unique behaviour combination. The algorithm is conceptually simple and effective at producing high-quality, diverse solutions, but it comes with a major limitation on its exploratory capabilities. With each additional behaviour, the set of solutions grows exponentially, making high-dimensional feature spaces infeasible. This thesis proposes an option for increasing behaviours with a novel Age-Layered MAP-Elites (ALME) algorithm where the population is separated into age layers and each layer has its own feature space. By using different behaviours in the different layers, the population migrates up through the layers experiencing selective pressure towards different behaviours. This algorithm is applied to a simulated intelligent agent environment to observe interesting emergent behaviours. It is observed that ALME is capable of producing a set of solutions with diversity in all behaviour dimensions while keeping the final population size low. It is also observed that ALME is capable of filling its top layer feature space more consistently than MAP-Elites with the same behaviour dimensions

Digital Ecosystems: Ecosystem-Oriented Architectures

We view Digital Ecosystems to be the digital counterparts of biological ecosystems. Here, we are concerned with the creation of these Digital Ecosystems, exploiting the self-organising properties of biological ecosystems to evolve high-level software applications. Therefore, we created the Digital Ecosystem, a novel optimisation technique inspired by biological ecosystems, where the optimisation works at two levels: a first optimisation, migration of agents which are distributed in a decentralised peer-to-peer network, operating continuously in time; this process feeds a second optimisation based on evolutionary computing that operates locally on single peers and is aimed at finding solutions to satisfy locally relevant constraints. The Digital Ecosystem was then measured experimentally through simulations, with measures originating from theoretical ecology, evaluating its likeness to biological ecosystems. This included its responsiveness to requests for applications from the user base, as a measure of the ecological succession (ecosystem maturity). Overall, we have advanced the understanding of Digital Ecosystems, creating Ecosystem-Oriented Architectures where the word ecosystem is more than just a metaphor.Comment: 39 pages, 26 figures, journa

Prospects for large-scale financial systems simulation

As the 21st century unfolds, we find ourselves having to control, support, manage or otherwise cope with large-scale complex adaptive systems to an extent that is unprecedented in human history. Whether we are concerned with issues of food security, infrastructural resilience, climate change, health care, web science, security, or financial stability, we face problems that combine scale, connectivity, adaptive dynamics, and criticality. Complex systems simulation is emerging as the key scientific tool for dealing with such complex adaptive systems. Although a relatively new paradigm, it is one that has already established a track record in fields as varied as ecology (Grimm and Railsback, 2005), transport (Nagel et al., 1999), neuroscience (Markram, 2006), and ICT (Bullock and Cliff, 2004). In this report, we consider the application of simulation methodologies to financial systems, assessing the prospects for continued progress in this line of research

Adaptive tension, self-organization and emergence : A complex system perspective of supply chain disruptions

The purpose of this thesis was to explore how microstate human interactions produce macro level self-organization and emergence in a supply disruption scenario, as well as discover factors and typical human behaviour that bring about disruptions. This study argues that the complex adaptive system’s view of complexity is most suited scholarly foundation for this research enquiry. Drawing on the dissipative structure based explanation of emergence and self-organization in a complex adaptive system, this thesis further argues that an energy gradient between the ongoing and designed system conditions, known as adaptive tension, causes supply chains to self-organize and emerge. This study adopts a critical realist ontology operationalized by a qualitative case research and grounded theory based analysis. The data was collected using repertory grid interviews of 22 supply chain executives from 21 firms. In all 167 cases of supply disruptions were investigated. Findings illustrate that agent behaviours like loss of trust, over ambitious pursuit, use of power and privilege, conspiring against best practices and heedless performance were contributing to disruption. Impacted by these behaviours, supply chains demonstrated impaired disruption management capabilities and increased disruption probability. It was also discovered that some of these system patterns and microstate agent behaviours pushed the supply chains to a zone of emergent complexity where these networks self-organized and emerged into new structures or embraced changes in prevailing processes or goals. A conceptual model was developed to explain the transition from micro agent behaviour to system level self-organization and emergence. The model described alternate pathways of a supply chain under adaptive tension. The research makes three primary research contributions. Firstly, based upon the theoretical model, this research presents a conceptualization of supply chain emergence and self-organization from dissipative structures and adaptive tension based view of complexity. Secondly, it formally introduces and validates the role of behavioural and cognitive element of human actions in a supply chain scenario. Lastly, it affirms the complex adaptive system based conceptualization of supply chain networks. These contributions succeed in providing organizations with an explanation for observed deviations in their operations performance using a behavioural aspect of human agents