Controlled design of human-like agents with context-guided learning for automated video game playing

Abstract

The video game industry state-of-the-practice ad hoc behaviour authoring techniques produce transparent and highly controllable autonomous agent artificial intelligence (AI) representations, but show limitations in adaptive and human-like behaviour design. Machine learning (ML) methods can cope with such constraints, but the black-box nature, high training costs in terms of data volume and time, as well as incompatibility with iterative workflows, make ML models unsuitable for commercial game development. To address the shortcomings of both these approaches, we investigate a non-disruptive, modular design approach to integrating small-scale learning models featuring performance and execution guarantees, as well as embedded human designer intent, into behaviour tree (BT) architecture for autonomous video game agents. We deployed the proposed design in the environment of a published, commercial video game 60 Seconds!, which we instrumented for agent training and evaluation using an off-the-shelf game engine, BT and a learning library. After quantitative analysis of the mass-scale gameplay telemetry dataset of 8,244,111 trajectories from real game users, we clustered the player population with respect to estimated play skill, using a gameplay context-based score metric. Output agent models were then developed and trained in the game’s environment by applying the design, guided by game context-relevant segmentation of logic and behaviour of the top play skill persona model, derived from the trajectory data of the 7% top-scoring player cluster. The output agent’s gameplay performance was benchmarked against that of a reference agent, and experimentally evaluated in a normalised game scenario against 18,947 human players. It was found to be valid in the context of the game environment, functional, and capable of pursuing gameplay objectives in unseen scenarios with competency. However, it was unable to outperform human players due to the suboptimal performance of its trained learning models. We determined that software stability issues of the learning library used, limited observation space, and egocentric data adversely affected agent training. While further work to improve the training process is necessary, the successful application of the context guided agent design in a commercial video game environment confirmed its potential for industrial applications. By contributing the design, the mass-scale dataset, and the tools used in our research, we enable the context-guided agents to be deployed in alternative contexts