3 research outputs found
NPC AI System Based on Gameplay Recordings
HĂ€sti optimeeritud mitte-mĂ€ngija tegelased (MMT) on vastaste vĂ”i meeskonna kaaslastena ĂŒheks peamiseks osaks mitme mĂ€ngija mĂ€ngudes. Enamus mĂ€nguroboteid on ehitatud jĂ€ikade sĂŒsteemide peal, mis vĂ”imaldavad vaid loetud arvu otsuseid ja animatsioone. Kogenud mĂ€ngijad suudavad eristada mĂ€nguroboteid inimmĂ€ngijatest ning ette ennustada nende liigutusi ja strateegiaid. See alandab mĂ€ngukogemuse kvaliteeti. SeetĂ”ttu, eelistavad mitme mĂ€ngijaga mĂ€ngude mĂ€ngijad mĂ€ngida pigem inimmĂ€ngijate kui MMTde vastu. Virtuaalreaalsuse (VR) mĂ€ngud ja VR mĂ€ngijad on siiani veel vĂ€ike osa mĂ€ngutööstusest ja mitme mĂ€ngija VR mĂ€ngud kannatavad mĂ€ngijabaasi kaotusest, kui mĂ€nguomanikud ei suuda leida teisi mĂ€ngijaid, kellega mĂ€ngida. See uurimus demonstreerib mĂ€ngulindistustel pĂ”hineva tehisintellekt (TI) sĂŒsteemi rakendatavust VR esimese isiku vaates tulistamismĂ€ngule Vrena. TeemamĂ€ng kasutab ebatavalist liikumisesĂŒsteemi, milles mĂ€ngijad liiguvad otsiankrute abil. VR mĂ€ngijate liigutuste imiteerimiseks loodi AI sĂŒsteem, mis kasutab mĂ€ngulindistusi navigeerimisandmetena. SĂŒsteem koosneb kolmest peamisest funktsionaalsusest. Need funktsionaalsused on mĂ€ngutegevuse lindistamine, andmete töötlemine ja navigeerimine. MĂ€ngu keskkond on tĂŒkeldatud kuubikujulisteks sektoriteks, et vĂ€hendada erinevate asukohal pĂ”hinevate olekute arvu ning mĂ€ngutegevus on lindistatud ajaintervallide ja tegevuste pĂ”hjal. Loodud mĂ€ngulogid on segmenteeritud logilĂ”ikudeks ning logilĂ”ikude abil on loodud otsingutabel. Otsingutabelit kasutatakse MMT agentide navigeerimiseks ning MMTde otsuste langetamise mehanism jĂ€ljendab olek-tegevus-tasu kontseptsiooni. Loodud töövahendi kvaliteeti hinnati uuringu pĂ”hjal, millest saadi mĂ€rkimisvÀÀrset tagasisidet sĂŒsteemi tĂ€iustamiseks.A well optimized Non-Player Character (NPC) as an opponent or a teammate is a major part of the multiplayer games. Most of the game bots are built upon a rigid system with numbered decisions and animations. Experienced players can distinguish bots from hu-man players and they can predict bot movements and strategies. This reduces the quality of the gameplay experience. Therefore, multiplayer game players favour playing against human players rather than NPCs. VR game market and VR gamers are still a small frac-tion of the game industry and multiplayer VR games suffer from loss of their player base if the game owners cannot find other players to play with. This study demonstrates the applicability of an Artificial Intelligence (AI) system based on gameplay recordings for a Virtual Reality (VR) First-person Shooter (FPS) game called Vrena. The subject game has an uncommon way of movement, in which the players use grappling hooks to navigate. To imitate VR playersâ movements and gestures an AI system is developed which uses gameplay recordings as navigation data. The system contains three major functionality. These functionalities are gameplay recording, data refinement, and navigation. The game environment is sliced into cubic sectors to reduce the number of positional states and gameplay is recorded by time intervals and actions. Produced game logs are segmented into log sections and these log sections are used for creating a look-up table. The lookup table is used for navigating the NPC agent and the decision mechanism followed a way similar to the state-action-reward concept. The success of the developed tool is tested via a survey, which provided substantial feedback for improving the system
Imitation learning through games: theory, implementation and evaluation
Despite a history of games-based research, academia has generally regarded
commercial games as a distraction from the serious business of AI, rather than as an
opportunity to leverage this existing domain to the advancement of our knowledge.
Similarly, the computer game industry still relies on techniques that were developed
several decades ago, and has shown little interest in adopting more progressive
academic approaches. In recent times, however, these attitudes have begun to change;
under- and post-graduate games development courses are increasingly common,
while the industry itself is slowly but surely beginning to recognise the potential
offered by modern machine-learning approaches, though games which actually
implement said approaches on more than a token scale remain scarce.
One area which has not yet received much attention from either academia or industry
is imitation learning, which seeks to expedite the learning process by exploiting data
harvested from demonstrations of a given task. While substantial work has been done
in developing imitation techniques for humanoid robot movement, there has been
very little exploration of the challenges posed by interactive computer games. Given
that such games generally encode reasoning and decision-making behaviours which
are inherently more complex and potentially more interesting than limb motion data,
that they often provide inbuilt facilities for recording human play, that the generation
and collection of training samples is therefore far easier than in robotics, and that
many games have vast pre-existing libraries of these recorded demonstrations, it is
fair to say that computer games represent an extremely fertile domain for imitation
learning research.
In this thesis, we argue in favour of using modern, commercial computer games to
study, model and reproduce humanlike behaviour. We provide an overview of the
biological and robotic imitation literature as well as the current status of game AI, highlighting techniques which may be adapted for the purposes of game-based
imitation. We then proceed to describe our contributions to the field of imitation
learning itself, which encompass three distinct categories: theory, implementation
and evaluation.
We first describe the development of a fully-featured Java API - the Quake2 Agent
Simulation Environment (QASE) - designed to facilitate both research and education
in imitation and general machine-learning, using the game Quake 2 as a testbed. We
outline our motivation for developing QASE, discussing the shortcomings of existing
APIs and the steps which we have taken to circumvent them. We describe QASEâs
network layer, which acts as an interface between the local AI routines and the
Quake 2 server on which the game environment is maintained, before detailing the
APIâs agent architecture, which includes an interface to the MatLab programming
environment and the ability to parse and analyse full recordings of game sessions.
We conclude the chapter with a discussion of QASEâs adoption by numerous
universities as both an undergraduate teaching tool and research platform.
We then proceed to describe the various imitative mechanisms which we have
developed using QASE and its MatLab integration facilities. We first outline a
behaviour model based on a well-known psychological model of human planning.
Drawing upon previous research, we also identify a set of believability criteria -
elements of agent behaviour which are of particular importance in determining the
âhumannessâ of its in-game appearance. We then detail a reinforcement-learning
approach to imitating the human playerâs navigation of his environment, centred
upon his pursuit of items as strategic goals. In the subsequent section, we describe
the integration of this strategic system with a Bayesian mechanism for the imitation
of tactical and motion-modelling behaviours. Finally, we outline a model for the
imitation of reactive combat behaviours; specifically, weapon-selection and aiming. Experiments are presented in each case to demonstrate the imitative mechanismsâ
ability to accurately reproduce observed behaviours.
Finally, we criticise the lack of any existing methodology to formally gauge the
believability of game agents, and observe that the few previous attempts have been
extremely ad-hoc and informal. We therefore propose a generalised approach to such
testing; the Bot-Oriented Turing Test (BOTT). This takes the form of an anonymous
online questionnaire, an accompanying protocol to which examiners should adhere,
and the formulation of a believability index which numerically expresses each agentâs
humanness as indicated by its observers, weighted by their experience and the
accuracy with which the agents were identified. To both validate the survey approach
and to determine the efficacy of our imitative models, we present a series of
experiments which use the believability test to evaluate our own imitation agents
against both human players and traditional artificial bots. We demonstrate that our
imitation agents perform substantially better than even a highly-regarded rule-based
agent, and indeed approach the believability of actual human players.
Some suggestions for future directions in our research, as well as a broader
discussion of open questions, conclude this thesis
Bayesian Imitation of Human Behavior in Interactive Computer Games
Gorman B, Thurau C, Bauckhage C, Humphrys M. Bayesian Imitation of Human Behavior in Interactive Computer Games. In: Proc. of the Int. Conf. on Pattern Recognition (ICPRâ06). Vol 1. IEEE; 2006: 1244-1247