Learning Social Preferences in Games

This paper presents a machine-learning approach to modeling human behavior in one-shot games. It provides a framework for representing and reasoning about the social factors that affect people’s play. The model predicts how a human player is likely to react to different actions of another player, and these predictions are used to determine the best possible strategy for that player. Data collection and evaluation of the model were performed on a negotiation game in which humans played against each other and against computer models playing various strategies. A computer player trained on human data outplayed Nash equilibrium and Nash bargaining computer players as well as humans. It also generalized to play people and game situations it had not seen before.Engineering and Applied Science

Learning models in interdependence situations

Many approaches to learning in games fall into one of two broad classes: reinforcement and belief learning models. Reinforcement learning assumes that successful past actions have a higher probability to be played in the future. Belief learning assumes that players have beliefs about which action the opponent(s) will choose and that players determine their own choice of action by finding the action with the highest payoff given the beliefs about the actions of others. Belief learning and (a specific type of) reinforcement learning are special cases of a hybrid learning model called Experience Weighted Attraction (EWA). Some previous studies explicitly state that it is difficult to determine the underlying process (either reinforcement learning, belief learning, or something else) that generated the data for several games. This leads to the main question of this thesis: Can we distinguish between different types of EWA-based learning, with reinforcement and belief learning as special cases, in repeated 2 x 2 games? In Chapter 2 we derive predictions for behavior in three types of games using the EWA learning model using the concept of stability: there is a large probability that all players will make the same choice in round t +1 as in t. Herewith, we conclude that belief and reinforcement learning can be distinguished, even in 2 x 2 games. Maximum differentiation in behavior resulting from either belief or reinforcement learning is obtained in games with pure Nash equilibria with negative payoffs and at least one other strategy combination with only positive payoffs. Our results help researchers to identify games in which belief and reinforcement learning can be discerned easily. Our theoretical results imply that the learning models can be distinguished after a sufficient number of rounds have been played, but it is not clear how large that number needs to be. It is also not clear how likely it is that stability actually occurs in game play. Thereto, we also examine the main question by simulating data from learning models in Chapter 3. We use the same three types of 2 x 2 games as before and investigate whether we can discern between reinforcement and belief learning in an experimental setup. Our conclusion is that this is also possible, especially in games with positive payoffs and in the repeated Prisoner’s Dilemma game, even when the repeated game has a relatively small number of rounds. We also show that other characteristics of the players’ behavior, such as the number of times a player changes strategy and the number of strategy combinations the player uses, can help differentiate between the two learning models. So far, we only considered "pure" belief and "pure" reinforcement learning, and nothing in between. For Chapter 4, we therefore consider a broader class of learning models and we try to find under which conditions, we can re-estimate three parameters of EWA learning model from simulated data, generated for different games and scenarios. The results show low rates of convergence of the estimation algorithm, and even if the algorithm converges then biased estimates of the parameters are obtained most of the time. Hence, we must conclude that re-estimating the exact parameters in a quantitative manner is difficult in most experimental setups. However, qualitatively we can find patterns that pinpoint in the direction of either belief or reinforcement learning. Finally, in the last chapter, we study the effect of a player’s social preferences on his own payoff in 2 x 2 games with only a mixed strategy equilibrium, under the assumption that the other player has no social preferences. We model social preferences with the Fehr-Schmidt inequity aversion model, which contains parameters for "envy" and "spite". Eighteen different mixed equilibrium games are identified that can be classified into Regret games, Risk games, and RiskRegret games, with six games in each class. The effects of envy and spite in these games are studied in five different status scenarios in which the player with social preferences receives much higher, mostly higher, about equal, mostly lower, or much lower payoffs. The theoretical and simulation results reveal that the effects of social preferences are variable across scenarios and games, even within scenario-game combinations. However, we can conclude that the effects of envy and spite are analogous, on average beneficial to the player with the social preferences, and most positive when the payoffs are about equal and in Risk games

Social Preference, Incomplete Information, and the Evolution of Ultimatum Game in the Small World Networks: An Agent-Based Approach

Certain social preference models have been proposed to explain fairness behavior in experimental games. Existing bodies of research on evolutionary games, however, explain the evolution of fairness merely through the self-interest agents. This paper attempts to analyze the ultimatum game's evolution on complex networks when a number of agents display social preference. Agents' social preference is modeled in three forms: fairness consideration or maintaining a minimum acceptable money level, inequality aversion, and social welfare preference. Different from other spatial ultimatum game models, the model in this study assumes that agents have incomplete information on other agents' strategies, so the agents need to learn and develop their own strategies in this unknown environment. Genetic Algorithm Learning Classifier System algorithm is employed to address the agents' learning issue. Simulation results reveal that raising the minimum acceptable level or including fairness consideration in a game does not always promote fairness level in ultimatum games in a complex network. If the minimum acceptable money level is high and not all agents possess a social preference, the fairness level attained may be considerably lower. However, the inequality aversion social preference has negligible effect on the results of evolutionary ultimatum games in a complex network. Social welfare preference promotes the fairness level in the ultimatum game. This paper demonstrates that agents' social preference is an important factor in the spatial ultimatum game, and different social preferences create different effects on fairness emergence in the spatial ultimatum game.Spatial Ultimatum Game, Complex Network, Social Preference, Agent Based Modeling

Network Analysis for Learners’ Concept Maps While Using Mobile Augmented Reality Gaming

Using mobile augmented reality games in education combines situated and active learning with pleasure. The aim of this research is to analyze the responses expressed by young, middle-aged, and elderly adults about the location-based mobile augmented reality (MAR) games using methods of content analysis, concept maps, and social network analysis (SNA). The responses to questions related to MAR game Ingress were collected from 36 adult players, aged 20–60, from Greece, and subsequently analyzed by means of content analysis, concept maps, and social network analysis. Our findings show that for question 1 (How do you feel when you endow the geographical space with personal preferences?), there was a differentiation of the answers between age groups with age groups agreeing in pairs, the first two and the last two, while for question 2 (Do you think that the game offers opportunities for learning and teaching geography, building on your previous geographical knowledge?), there was an overlap in responses of participants among age groups. It was also revealed that the MAR games foster a constructivism approach of learning, as their use learning becomes an active, socially supported process of knowledge construction

The Evolution of ‘Theory of Mind:’ Theory and Experiments

This paper provides an evolutionary foundation for our capacity to attribute preferences to others. This ability is intrinsic to game theory, and is a key component of “Theory of Mind,” perhaps the capstone of social cognition. We argue here that this component of theory of mind allows organisms to efficiently modify their behavior in strategic environments with a persistent element of novelty. Such environments are represented here by multistage games of perfect information with randomly chosen outcomes. “Theory of Mind” then yields a sharp, unambiguous advantage over less sophisticated, behavioral approaches to strategic interaction. In related experiments, we show the subscale for social skills in standard tests for autism is a highly significant determinant of the speed of learning in such games

How to Design Game-based Healthcare Applications for Children? - A Study on Children’s Game Preferences

Game-based design can be used to develop engaging health applications for children. This engagement can only be realised when design is tailored to their preferences. In this study we investigate game preferences of children and translate these into design recommendations. Game preferences of children aged 6 to 12 were assessed through a questionnaire. Outcomes were classified by means of the 7D framework which divides game content into seven linear domains. Significant differences in mean scores among demographic subgroups were explored. Sixty-five children participated (M=9 years, SD=0.24, 36 boys, 29 girls, 8 children with asthma). Data showed high preference for content in domains novelty (Mnovelty=63) and dedication (Mdedication=70). Analysis resulted in subdivision of scores based on gender, age and playing frequency. Striking differences in scores were found between boys and girls in discord (Mboys=62, Mgirls=19), intensity (Mboys=60, Mgirls=27), rivalry (Mboys=53, Mgirls=31) and threat (Mboys=64, Mgirls=25). To design games for children we recommend to stimulate curiosity by offering variation and discovery, to enable achievement, learning and social contact. A divergence in preferences for boys and girls must be regarded. Opposed to boys, girls may lose interest in games that have violent or scary content, that are mainly competitive or demand continuous effort

Identifying Player Types to Tailor Game-Based Learning Design to Learners:Cross-sectional Survey using Q Methodology

BACKGROUND: Game-based learning appears to be a promising instructional method because of its engaging properties and positive effects on motivation and learning. There are numerous options to design game-based learning; however, there is little data-informed knowledge to guide the choice of the most effective game-based learning design for a given educational context. The effectiveness of game-based learning appears to be dependent on the degree to which players like the game. Hence, individual differences in game preferences should be taken into account when selecting a specific game-based learning design. OBJECTIVE: We aimed to identify patterns in students' perceptions of play and games-player types and their most important characteristics. METHODS: We used Q methodology to identify patterns in opinions on game preferences. We recruited undergraduate medical and dental students to participate in our study and asked participants to sort and rank 49 statements on game preferences. These statements were derived from a prior focus group study and literature on game preferences. We used by-person factor analysis and varimax rotation to identify common viewpoints. Both factors and participants' comments were used to interpret and describe patterns in game preferences. RESULTS: From participants' (n=102) responses, we identified 5 distinct patterns in game preferences: the social achiever, the explorer, the socializer, the competitor, and the troll. These patterns revolved around 2 salient themes: sociability and achievement. The 5 patterns differed regarding cheating, playing alone, story-telling, and the complexity of winning. CONCLUSIONS: The patterns were clearly interpretable, distinct, and showed that medical and dental students ranged widely in how they perceive play. Such patterns may suggest that it is important to take students' game preferences into account when designing game-based learning and demonstrate that not every game-based learning-strategy fits all students. To the best of our knowledge, this study is the first to use a scientifically sound approach to identify player types. This can help future researchers and educators select effective game-based learning game elements purposefully and in a student-centered way

JEU PÉDAGOGIQUE ET APPRENTISSAGE COOPÉRATIF : ÉTUDE FRANCO-ANGLOPHONE DES PRATIQUES DE FORMATION DE TRAVAILLEURS SOCIAUX

International audienceThis article presents the first results from an exploratory research enquiry about the usage of learning games by teachers and instructors of social workers. Two forms of play inspired learning are dominant in the discourse of the teachers interviewed and in the literature reviewed: open simulation games with non predefined roles; creative and artistic activities. These teaching practices are valued for the transmission of individual competencies that form a basis for cooperation, but learning games are not often used as an exercise of cooperation in their own right. These tendencies and teaching preferences seem to be common to French and English speaking teachers and are anchored in a strong professionnal value system. Other approaches and techniques that could be of interest for this field seem to be neglected. The author suggests possible directions for developing learning games in this context.Cet article rend compte des premiers résultats d’une recherche exploratoire sur les usages du jeu pédagogique par les formateurs et enseignants en formation initiale de travailleurs sociaux. Deux approches ludiques sont largement citées dans le discours des interviewés et dans la littérature professionnelle: les jeux de simulation à caractère non formalisé ou « libre » ; les jeux et activités à caractère artistique ou créatif. Ces pratiques pédagogiques sont valorisées pour le développement de la connaissance de soi, l’acquisition d’une posture professionnelle et l’accès à une meilleure compréhension de l’altérité. Ainsi, le jeu pédagogique est valorisé pour la transmission des compétences individuelles nécessaires pour coopérer, mais il est peu sollicité en tant qu’exercice de coopération à part entière. Ces orientations pédagogiques, qui semblent être partagées par les francophones et anglophones, peuvent être rapprochées des valeurs fondatrices du travail social, mais laissent de côté d’autres approches et techniques pourtant intéressants pour ce champ. L’auteur suggère des pistes pour élargir la portée du jeu pédagogique dans ce contexte