Hybridisation for versatile decision-making in game opponent A

Doughty, Mark

Fowler, Martin

Hunter, Andrew

English

University of Lincoln Institutional Repository

Hybridisation for versatile decision-making in  Game Opponent AI  Martin Fowler, Andrew Hunter and Mark Doughty  Faculty of Applied Computing Science University of Lincoln Brayford Pool Lincoln  mfowler@lincoln.ac.uk    Abstract - Traditionally game AI has been implemented using rule-based approaches augmented by novel algorithms. The resultant predictability of such approaches can diminish the efficacy and longevity of the AI opponents. This research will investigate a hybrid approach to develop better opponents leading to more flexible decision making and ultimately to a more satisfying game experience.  Keywords  Artificial Intelligence, Games, Battle Simulation  I. INTRODUCTION  Ancient battle has become the focus of a significant genre of computer game often referred to as Real Time Strategy (RTS). These games offer the player control over an army and allow them to control tactics to reach their strategic goals. These games can be played either as a single-player experience against an AI opponent or against another human, usually via a remote link such as an Internet server.  Traditional approaches in game AI rely heavily on rule-based systems. Such approaches offer a stable way to develop AI but the resultant implementation often suffers for two reasons. To cover all situations the rule base must be large which leads to long search times. Also rules can only be applied in specific circumstances, all of which must be identified by the developer.  Opponent AI developed in this way can challenge the novice player but once the behaviour has been observed over the course of a small number of games a good human player can overcome it with relative ease and in some cases subvert its inflexibility to beat it.  The single-player experience can be viewed as a training exercise before the player meets their real human opponent but the experience of playing against predictable, rule-based AI will be very different than taking on a human. To bridge this gap and also make the single-player experience more rewarding this research intends to add flexibility to the traditional rule-based approach thus diminishing the predictability of the AI. This paper is laid out as follows. In section two a cognitive modelling approach is discussed. Section three lays out the methods under investigation which can help the agent to manage uncertainty. Section four details the different categories of rules required. Section five considers the methods available for performing fuzzy inference within the context of this research. Section six explains the proposed strategy for the agent to learn and adapt. Section seven describes a method for assessing the efficacy of the agent.  II. COGNITIVE MODELLING  Although Real-Time Strategy games involve many agents this research centres on the combat manager agent[1] or 'General' which has responsibility for the overall control of an army on a battlefield. Cognitive modelling techniques will be used to identify the characterics required to build an effective opponent agent of this type.  Cognitive and Mental Modelling have been used in the study of Psychology for decades as a medium to describe the processes required to solve specific problems to achieve a goal. Recent work [2] into cognitive modelling in games AI has sought to adapt such techniques to aid the creation of realistic agents in virtual worlds. Funge puts cognitive modelling at the top of his conceptual hierarchy of AI. The layers of this hierarchy can be expressed as the key questions designed to be solved by that layer of the AI.  Cognitive Modelling What should the character do next? Behavioural What is the character doing at the moment? Physical Where is the character? What posture is the character assuming? Kinematic Which shape is used to represent the character?  Geometric What triangles are drawn? Where are they drawn?  The scope of this research lies within the top two layers of Funge's hierarchy: Cognitive Modelling and Behaviour. The 'General' agent will have no embodiment in the environment but will be a controlling influence on an army of less autonomous agents. Figure one shows Funge's conceptual view of game AI.     Figure 1. Cognitive Modelling Hierarchy  It is proposed that all behaviour rules will be modelled using Funge's Cognitive Modelling Language (CML). CML relates closely to existing programming languages such as C/C++. This allows for closer semantic accuracy from the candidate rules identified and modelled in CML to the expert rule set used as the basis for the Expert System. CML is an integral part of an overall method. Funge offers an academic reflection on key themes in game AI and intends to offer a method of specifying rules at a higher level of abstraction than normal.  Funge's work offers a greater understanding of the bridge between simple behaviour and more complex goal-oriented planning. However the inability to evolve the rule set to cope with a new situation leaves a gap between game AI technique and an effective cognitive model of a decision maker. CML is currently based on naïve set theory and first-order logic.  III. MANAGING UNCERTAINTY  Rule based approaches enable the AI to respond to specific situations but no two games are alike. This research will investigate how other established AI techniques can allow the agent to respond to circumstance not covered by existing rules, or to learn to apply the rules to different situations.   A. Fuzzy Logic  When Aristotle first described the theory of logic as being bivalent he said any statement made is either true or false. Lotfi Zadeh offered an alternative to bivalent logic when he proposed the theory of fuzzy logic and fuzzy sets. Although Zadeh was not the first to propose the use of multivalent logic he is usually cited in computer science articles as its pioneer because of the application of his work in AI. Fuzzy logic differs from 'Crisp' logic in its use of linguistic variables alongside the quantative variables. Some examples of linguistic variables are: very, slightly, near, far, more and less. Such variables have no absolute threshold or value but provide an expression of the degree of membership of a fuzzy set.  Fuzzy set theory contrasts with naive set theory with the idea of degrees of membership. In naive set theory things are either members of a set or not, so membership can be expressed as 1 for members and 0 for non-members; true or false. With a fuzzy set everything is a member of the set but by degrees ranging from 0 to 1 and can be expressed with the linguistic variables described above.  Fuzzy logic allows rules to be applied by degrees rather than absolutes; shades of grey rather than black and white.  Reference [3] lays out criteria to assess whether a system might benefit from fuzzy logic. Tactical decision-making as undertaken by an autonomous agent such as the 'General' matches these criteria closely. Specifically a cognitive model of tactical-decision making is complex, vague and has continuous input and output. The use of fuzzy logic alongside a rule-based approach should allow those rules to be applied flexibly and in new and different situations.  Reference [4] gives a description of a limited implementation of fuzzy logic in game AI. The controllers describe are not fully fuzzy due to expected performance issues. The main restriction is that only one output behaviour is allowed. The 'General' controls an army and is therefore capable of ordering multiple behaviours in parallel by dividing the available forces; for example some troops could be ordered to attack the left flank of the enemy army while another division defends the camp.  B. Fuzzy Cognitive Map (FCM)  Reference [5] describes an approach Fuzzy Cognitive Map which combines elements of Fuzzy Logic and Artificial Neural Networks. FCMs offer a simple method of modelling and testing causal relationships between different concepts. The FCM is initially drawn by an expert in the specific problem domain. Once the FCM is deployed it can be used to monitor and refine those causal relationships. FCM is likely to be more useful as an initial step in identifying candidate rules than a means to create the flexible decision-maker as this research intends.  C. Hybrid Fuzzy-Expert System  The disadvantages of classic expert systems include: difficulty in identifying a sufficient rule set, the complexity of modelling those rules in a computational way and the inability of such a system to refine itself and to create better rules. Command and control can be built effectively using a fuzzy rule-based system. The rules will be applied after going through a 'Fuzzification' process.  One advantage of this approach is the requirement for a smaller rule set since any rules are applied in less definitive ways broadening their applicability.  Fuzzy Logic is described as means to represent commonsense knowledge [6] and it is this flexible way humans apply their knowledge they gain through experience that a hybridised system attempts to emulate. A fuzzy Cognitive Modelling Behavioural PhysicalKinematicGeometricThe agent must continually assess the threat posed by the enemy, his continued success in commanding his army to deal with such a threat, the achievement of his tactical goals and any environmental factors which may affect the course of the battle. Rules governing which combination of factors trigger a change in goals or overall strategy would be included in this category. system is a type of expert system [5] since each fuzzy rule mirrors a vague rule applied by an expert.  Reference [7] offers a five stage process in the design of a Hybrid Fuzzy-Expert System. This starts with analysis of the problem domain from which linguistic variables, fuzzy sets and fuzzy rules are elicited. These are used to perform fuzzy inference on the Expert rules. Learning exercises are then performed on the system to allow for fine-tuning of fuzzy membership values.  D. Application of Rules  During the simulation it is expected that these rules will be applied iteratively in reaction to enemy movement and strategy.  A division of troops will be made by the agent and these divisions will draw up in one of the standard formations used during the historical period chosen.  Each division will be given a position to occupy.  This provides the less complex troop AI a set of goals to perform. Once all troops are in formation the army can advance to take new ground or hold their current position.  IV. BEHAVIOUR RULES  The control of tactics in war games is controlled by a set of behaviour rules. The rules are predefined, received logic divided into goals and actions pertaining to a given situation or predicate.  The rules for the Expert System will be mainly derived from two sources. Initially the study of military literature will provide tactical analysis of ancient battles such as troop formations and a narrative of manoeuvres. This study should inform the design of an interface allowing human participation. Secondly observation and knowledge elicitation from a selection of human players. These players will be allowed to control armies within the simulation environment and the heuristics they use will augment the rules derived from military literature.  E. Enhanced CML  If CML can be extended to allow for the expression of fuzzy logic such a change would facilitate the expression of more human-like cognitive behaviour rules. The study of literature on FCM may help to combine CML with Fuzzy Logic.   V. FUZZY INFERENCE The specification of such rules depends on the diversity of factors which can be manipulated by the 'General' and also by any external factors which could affect the successful execution of any order given by the 'General'. Rules and actions for the Expert System can be placed in one of three categories.  There are two main methods used for fuzzy inference [7]: Mamdani and Sugeno. Mamdani offers a more intuitive rule definition leading to more transparent application of rules. The price of this transparency is a high computational cost. To offset this Mamdani will only be used during training exercises where an understanding of changes to the rules is required. A less computationally intensive method will be employed once training is complete to allow further adapt ion of the rules once the 'General' faces a real opponent. There are four stages to Fuzzy Inference [7]: Fuzzification, Rule Evaluation, Aggregation and Defuzzification.  A. Position and Formation  Rules in this category deal with how the divisions of the army into fighting groups, their position on the battlefield and what formation the fighting groups take. Figure Two shows two of the formations which will can be used depending on the tactical goals identified by the 'General'.  A. Fuzzification    The degree of membership to which any 'Crisp' input belongs within the appropriate fuzzy set is evaluated. Linguistic variables provide subsets where the input value will reside. Since these subsets usually overlap it is possible for the input value to reside within two subsets.  B. Rule Evaluation    The rules relating to the specific input values are evaluated and the rules pertaining to particular fuzzy subsets are 'fired' to produce a relevant output value. Output values are 'clipped' and 'scaled' according to the level of truth of the rule which provided it. Figure 2. Typical Troop Formations  B. Engagement and Manoeuvre  Rules in this category deal with the goals of each fighting group as regards engagement in attack and defence. Such rules would also include the manoeuvre of the fighting groups in order to gain the advantage of superior position.  C. Aggregation   All rule outputs are combined to give a single fuzzy set for each output. C. Assessment   D. Defuzzification  Fuzzy Inference is used to evaluate the rules but the final output must again be a 'Crisp' number.  VI. LEARNING STRATEGY AND COOPERATIVE EVOLUTION  The 'General' will be equipped with expert rules to guide its decision making process. To become a more flexible tactician the 'General' must be able to change and evolve the rules and behaviour determining which strategy to employ in a given situation. To make a more competent opponent it would be beneficial for the 'General' to learn and adapt to the opposition strategy. This is not only relevant to a single battle but if the 'General' stores what it has learnt and adds to that learning each time it fights the diversity of knowledge should likewise expand. This should lead to the 'General' AI learning and therefore undermining the strategy of the human player it plays most often, leading to symbiotic learning between the human and computer. The human player will change strategy as the current one becomes ineffective, and the AI will evolve to beat the new strategy developed by the human.  This development of difficulty levels is in contrast to the current way AI difficulty is handled in most RTS games where the human learns to beat the computer and upon gaining the upper hand will set the game to play at a higher difficulty level until the highest difficulty level is surpassed.  Two agents will fight set battles against each other. One will be in teaching mode where the rules will remain stable. The opponent will play in training mode. Rules will be passed to the trainee from the teacher. The trainee will use the hybrid approach and 'fuzzify' the rules during the battle. Changes to membership weighting will remain local to the trainee until a set number of battles have been fought. The roles are then reversed and the new trainee inherits the adjusted membership weighting.  VII. ASSESSING THE AGENT  In computer games the autonomous agent will often follow a detailed script and behave predictably, making them appear less human. Emulating human behaviour in autonomous agents should make them less predictable. Alan Turing's famous test [8] was for decades the benchmark for Artificial Intelligence. Although now regarded as an incomplete indicator of intelligence, Turing's test can be adapted to assess character emulation within the behavioural confines of a war game.   In brief Turing's claims can be paraphrased as what 'acts' intelligently is intelligent. The 'General' will be designed to 'act' intelligently and so enhance the game experience. The purpose of this research project is to produce a decision-making agent which offers a more varied, less predictable experience for the game player. Since the decision making agent will 'live' in a bespoke environment it will not be possible to match its abilities against a market leading AI system. It is, however, possible to view how successful the fuzzy system is when faced with the crisp logic, rule-based counterpart.  VIII. CONCLUSION  The hybridisation of a traditional rule-based approach along with fuzzy logic offers an approach new to games. This approach may improve the AI enough that the single-player experience becomes more rewarding to play and worthwhile as a training exercise prior to taking on unpredictable human opponents.  The next stage in the research is to create virtual environments capable of containing the simulation experiments needed to test the AI. The 'General' must not only be able to reason about the opposing army but must also decide on the strategic use of terrain.  REFERENCES  [1] Bob Scott (2002): "Architecting an RTS AI". In: Steve Rabin (ed.): AI Game Programming Wisdom. Charles River Media: Massachusetts, pp 397-401 [2] John David Funge (1999): AI for Games and Animation - A Cognitive Modelling Approach. A. K. Peters: Massachusetts [3] F. Martin McNeill and Ellen Thro (1994): Fuzzy Logic - A Practical Approach. Academic Press: London [4] Thor Alexander (2002): "An Optimised Fuzzy Logic Architecture for Decision-Making". In: Steve Rabin (ed.): AI Game Programming Wisdom. Charles River Media: Massachusetts, pp 367-374 [5] Bart Kosko (1994): Fuzzy Thinking: The New Science Of Fuzzy Logic. Flamingo: London [6] Lotfi Zadeh (1987): "Commonsense and Fuzzy Logic" In: Nick Cercone and Gordon McCalla (ed.): The Knowledge Frontier - Essays in the representation of Knowledge. Springer-Verlag: New York, pp 103-136 [7] Michael Negnevitsky (2002): Artificial Intelligence - A Guide to Intelligent Systems. Addison-Wesley: London [8] Janet Finlay and Alan Dix (1996): An Introduction to Artificial Intelligence. UCL Press: London  

Hybridisation for versatile decision-making in game opponent AI

Abstract

Similar works

Full text

Available Versions

University of Lincoln Institutional Repository