The Challenge of Believability in Video Games: Definitions, Agents Models and Imitation Learning

In this paper, we address the problem of creating believable agents (virtual characters) in video games. We consider only one meaning of believability, ``giving the feeling of being controlled by a player'', and outline the problem of its evaluation. We present several models for agents in games which can produce believable behaviours, both from industry and research. For high level of believability, learning and especially imitation learning seems to be the way to go. We make a quick overview of different approaches to make video games' agents learn from players. To conclude we propose a two-step method to develop new models for believable agents. First we must find the criteria for believability for our application and define an evaluation method. Then the model and the learning algorithm can be designed

Evolutionary games on graphs

Game theory is one of the key paradigms behind many scientific disciplines from biology to behavioral sciences to economics. In its evolutionary form and especially when the interacting agents are linked in a specific social network the underlying solution concepts and methods are very similar to those applied in non-equilibrium statistical physics. This review gives a tutorial-type overview of the field for physicists. The first three sections introduce the necessary background in classical and evolutionary game theory from the basic definitions to the most important results. The fourth section surveys the topological complications implied by non-mean-field-type social network structures in general. The last three sections discuss in detail the dynamic behavior of three prominent classes of models: the Prisoner's Dilemma, the Rock-Scissors-Paper game, and Competing Associations. The major theme of the review is in what sense and how the graph structure of interactions can modify and enrich the picture of long term behavioral patterns emerging in evolutionary games.Comment: Review, final version, 133 pages, 65 figure

Programming Robosoccer agents by modelling human behavior

The Robosoccer simulator is a challenging environment for artificial intelligence, where a human has to program a team of agents and introduce it into a soccer virtual environment. Most usually, Robosoccer agents are programmed by hand. In some cases, agents make use of Machine learning (ML) to adapt and predict the behavior of the opposite team, but the bulk of the agent has been preprogrammed. The main aim of this paper is to transform Robosoccer into an interactive game and let a human control a Robosoccer agent. Then ML techniques can be used to model his/her behavior from training instances generated during the play. This model will be used later to control a Robosoccer agent, thus imitating the human behavior. We have focused our research on low-level behavior, like looking for the ball, conducting the ball towards the goal, or scoring in the presence of opponent players. Results have shown that indeed, Robosoccer agents can be controlled by programs that model human play.Publicad

Correcting and improving imitation models of humans for Robosoccer agents

Proceeding of: 2005 IEEE Congress on Evolutionary Computation (CEC'05),Edimburgo, 2-5 Sept. 2005The Robosoccer simulator is a challenging environment, where a human introduces a team of agents into a football virtual environment. Typically, agents are programmed by hand, but it would be a great advantage to transfer human experience into football agents. The first aim of this paper is to use machine learning techniques to obtain models of humans playing Robosoccer. These models can be used later to control a Robosoccer agent. However, models did not play as smoothly and optimally as the human. To solve this problem, the second goal of this paper is to incrementally correct models by means of evolutionary techniques, and to adapt them against more difficult opponents than the ones beatable by the human.Publicad

On the coexistence of cooperators, defectors and conditional cooperators in the multiplayer iterated Prisoner's Dilemma

Recent experimental evidence [Gruji\'c et al., PLoS ONE 5, e13749 (2010)] on the spatial Prisoner's Dilemma suggests that players choosing to cooperate or not on the basis of their previous action and the actions of their neighbors coexist with steady defectors and cooperators. We here study the coexistence of these three strategies in the multiplayer iterated Prisoner's Dilemma by means of the replicator dynamics. We consider groups with n = 2, 3, 4 and 5 players and compute the payoffs to every type of player as the limit of a Markov chain where the transition probabilities between actions are found from the corresponding strategies. We show that for group sizes up to n = 4 there exists an interior point in which the three strategies coexist, the corresponding basin of attraction decreasing with increasing number of players, whereas we have not been able to locate such a point for n = 5. We analytically show that in the infinite n limit no interior points can arise. We conclude by discussing the implications of this theoretical approach on the behavior observed in experiments.Comment: 12 pages, 10 figures, uses elsart.cl

Grouping promotes both partnership and rivalry with long memory in direct reciprocity

Biological and social scientists have long been interested in understanding how to reconcile individual and collective interests in iterated Prisoner's Dilemma. Many effective strategies have been proposed, and they are often categorized into one of two classes, `partners' and `rivals.' More recently, another class, `friendly rivals,' has been identified in longer-memory strategy spaces. Friendly rivals qualify as both partners and rivals: They fully cooperate with themselves, like partners, but never allow their co-players to earn higher payoffs, like rivals. Although they have appealing theoretical properties, it is unclear whether they would emerge in evolving population because most previous works focus on memory-one strategy space, where no friendly rival strategy exists. To investigate this issue, we have conducted large-scale evolutionary simulations in well-mixed and group-structured populations and compared the evolutionary dynamics between memory-one and memory-three strategy spaces. In a well-mixed population, the memory length does not make a major difference, and the key factors are the population size and the benefit of cooperation. Friendly rivals play a minor role because being a partner or a rival is often good enough in a given environment. It is in a group-structured population that memory length makes a stark difference: When memory-three strategies are available, friendly rivals become dominant, and the cooperation level nearly reaches a maximum, even when the benefit of cooperation is so low that cooperation would not be achieved in a well-mixed population. This result highlights the important interaction between group structure and memory lengths that drive the evolution of cooperation.Comment: 18 pages, 11 figure

Comparing dynamitic difficulty adjustment and improvement in action game

A thesis submitted to the University of Bedfordshire in partial fulfilment of the requirements for the degree of Master ResearchDesigning a game difficulty is one of the key things as a game designer. Player will be feeling boring when the game designer makes the game too easy or too hard. In the past decades, most of single player games can allow players to choose the game difficulty either easy, normal or hard which define the overall game difficulty. In action game, these options are lack of flexibility and they are unsuitable to the player skill to meet the game difficulty. By using Dynamic Difficulty Adjustment (DDA), it can change the game difficulty in real time and it can match different player skills. In this paper, the final goal is the comparison of the three DDA systems in action game and apply an improved DDA. In order to apply a new improved DDA, this thesis will evaluate three chosen DDA systems with chosen action decision based AI for action game. A new DDA measurement formula is applied to the comparing section

Evolutionary instability of selfish learning in repeated games

Across many domains of interaction, both natural and artificial, individuals use past experience to shape future behaviors. The results of such learning processes depend on what individuals wish to maximize. A natural objective is one’s own success. However, when two such “selfish” learners interact with each other, the outcome can be detrimental to both, especially when there are conflicts of interest. Here, we explore how a learner can align incentives with a selfish opponent. Moreover, we consider the dynamics that arise when learning rules themselves are subject to evolutionary pressure. By combining extensive simulations and analytical techniques, we demonstrate that selfish learning is unstable in most classical two-player repeated games. If evolution operates on the level of long-run payoffs, selection instead favors learning rules that incorporate social (other-regarding) preferences. To further corroborate these results, we analyze data from a repeated prisoner’s dilemma experiment. We find that selfish learning is insufficient to explain human behavior when there is a trade-off between payoff maximization and fairness