Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player Characters using Reinforcement Learning

In this paper, we introduce a skill-balancing mechanism for adversarial non-player characters (NPCs), called Skilled Experience Catalogue (SEC). The objective of this mechanism is to approximately match the skill level of an NPC to an opponent in real-time. We test the technique in the context of a First-Person Shooter (FPS) game. Specifically, the technique adjusts a reinforcement learning NPC's proficiency with a weapon based on its current performance against an opponent. Firstly, a catalogue of experience, in the form of stored learning policies, is built up by playing a series of training games. Once the NPC has been sufficiently trained, the catalogue acts as a timeline of experience with incremental knowledge milestones in the form of stored learning policies. If the NPC is performing poorly, it can jump to a later stage in the learning timeline to be equipped with more informed decision-making. Likewise, if it is performing significantly better than the opponent, it will jump to an earlier stage. The NPC continues to learn in real-time using reinforcement learning but its policy is adjusted, as required, by loading the most suitable milestones for the current circumstances.Comment: IEEE Conference on Computational Intelligence and Games (CIG). August 201

Opponent awareness at all levels of the multiagent reinforcement learning stack

Multiagent Reinforcement Learning (MARL) has experienced numerous high profile successes in recent years in terms of generating superhuman gameplaying agents for a wide variety of videogames. Despite these successes, MARL techniques have failed to be adopted by game developers as a useful tool to be used when developing their games, often citing the high computational cost associated with training agents alongside the difficulty of understanding and evaluating MARL methods as the two main obstacles. This thesis attempts to close this gap by introducing an informative modular abstraction under which any Reinforcement Learning (RL) training pipeline can be studied. This is defined as the MARL stack, which explicitly expresses any MARL pipeline as an environment where agents equipped with learning algorithms train via simulated experience as orchestrated by a training scheme. Within the context of 2-player zero-sum games, different approaches at granting opponent awareness at all levels of the proposed MARL stack are explored in broad study of the field. At the level of training schemes, a grouping generalization over many modern MARL training schemes is introduced under a unified framework. Empirical results are shown which demonstrate that the decision over which sequence of opponents a learning agent will face during training greatly affects learning dynamics. At the agent level, the introduction of opponent modelling in state-of-the art algorithms is explored as a way of generating targeted best responses towards opponents encountered during training, improving upon the sample efficiency of these methods. At the environment level the use of MARL as a game design tool is explored by using MARL trained agents as metagame evaluators inside an automated process of game balancing

Eye quietness and quiet eye in expert and novice golf performance: an electrooculographic analysis

Quiet eye (QE) is the final ocular fixation on the target of an action (e.g., the ball in golf putting). Camerabased eye-tracking studies have consistently found longer QE durations in experts than novices; however, mechanisms underlying QE are not known. To offer a new perspective we examined the feasibility of measuring the QE using electrooculography (EOG) and developed an index to assess ocular activity across time: eye quietness (EQ). Ten expert and ten novice golfers putted 60 balls to a 2.4 m distant hole. Horizontal EOG (2ms resolution) was recorded from two electrodes placed on the outer sides of the eyes. QE duration was measured using a EOG voltage threshold and comprised the sum of the pre-movement and post-movement initiation components. EQ was computed as the standard deviation of the EOG in 0.5 s bins from –4 to +2 s, relative to backswing initiation: lower values indicate less movement of the eyes, hence greater quietness. Finally, we measured club-ball address and swing durations. T-tests showed that total QE did not differ between groups (p = .31); however, experts had marginally shorter pre-movement QE (p = .08) and longer post-movement QE (p < .001) than novices. A group × time ANOVA revealed that experts had less EQ before backswing initiation and greater EQ after backswing initiation (p = .002). QE durations were inversely correlated with EQ from –1.5 to 1 s (rs = –.48 - –.90, ps = .03 - .001). Experts had longer swing durations than novices (p = .01) and, importantly, swing durations correlated positively with post-movement QE (r = .52, p = .02) and negatively with EQ from 0.5 to 1s (r = –.63, p = .003). This study demonstrates the feasibility of measuring ocular activity using EOG and validates EQ as an index of ocular activity. Its findings challenge the dominant perspective on QE and provide new evidence that expert-novice differences in ocular activity may reflect differences in the kinematics of how experts and novices execute skills

Being Jacques Villeneuve: Formula One, 'Agency' and the Fan

DVD disc of supplementary material available with the print copy of this thesis, held at the University of Waikato Library.In this thesis, I analyse my fandom for the Formula One driver, Jacques Villeneuve. Despite its rampant commercialism, innovative mediation, prestige and popular status within global sport, Formula One is surprisingly an under-researched topic in academia. Moreover, 'intense' fandom has often been stigmatised; at worst associating such individuals with pathological and obsessive behaviours or refuting their affections as merely symptomatic of the socio-economic forces that transform fans into duped consumers. This thesis argues against such simplistic disqualifications and reconceptualises fandom in light of how the structure/agency binary has itself been reconceptualised within media and cultural studies. Rather than privileging either the determining social, mediated and commercial structures, or championing the 'active agential' capacities of social individuals, Grossberg's notions of 'affect' and 'structured mobility' are drawn upon to underpin a more flexible explanation of contemporary fandom. In particular, affect offers theoretical purchase for how fans form attachments with selective media objects and why these come to 'matter' for specific individuals. Furthermore, by marrying affect with 'structured mobility', affective investments are recognised for their capacity to 'anchor' individuals in specific and concrete spatial/temporal 'moments' of social reality as they navigate both the mediated apparatus of the sport and the structured social, cultural and economic terrain that shapes their mediated fandom. Such insights are developed through a 'funnelling' approach in this thesis which moves from an examination of collective Formula One fandom to my own, exploring the affective traces of a friction that Villeneuve's maverick status provided within the broader machinery of the sport and to which this fan has responded

The Society of the Selfie

This book explores how the Internet is connected to the global crisis of liberal democracy. Today, self-promotion is at the heart of many human relationships. The selfie is not just a social media gesture people love to hate. It is also a symbol of social reality in the age of the Internet. Through social media people have new ways of rating and judging themselves and one another, via metrics such as likes, shares, followers and friends. There are new thirsts for authenticity, outlets for verbal aggression, and social problems. Social media culture and neoliberalism dovetail and amplify one another, feeding social estrangement. With neoliberalism, psychosocial wounds are agitated and authoritarianism is provoked. Yet this new sociality also inspires resistance and political mobilisation. Illustrating ideas and trends with examples from news and popular culture, the book outlines and applies theories from Debord, Foucault, Fromm, Goffman, and Giddens, among others. Topics covered include the global history of communication technologies, personal branding, echo chamber effects, alienation and fear of abnormality. Information technologies provide channels for public engagement where extreme ideas reach farther and faster than ever before, and political differences are widened and inflamed. They also provide new opportunities for protest and resistance

The Society of the Selfie

This book explores how the Internet is connected to the global crisis of liberal democracy. Today, self-promotion is at the heart of many human relationships. The selfie is not just a social media gesture people love to hate. It is also a symbol of social reality in the age of the Internet. Through social media people have new ways of rating and judging themselves and one another, via metrics such as likes, shares, followers and friends. There are new thirsts for authenticity, outlets for verbal aggression, and social problems. Social media culture and neoliberalism dovetail and amplify one another, feeding social estrangement. With neoliberalism, psychosocial wounds are agitated and authoritarianism is provoked. Yet this new sociality also inspires resistance and political mobilisation. Illustrating ideas and trends with examples from news and popular culture, the book outlines and applies theories from Debord, Foucault, Fromm, Goffman, and Giddens, among others. Topics covered include the global history of communication technologies, personal branding, echo chamber effects, alienation and fear of abnormality. Information technologies provide channels for public engagement where extreme ideas reach farther and faster than ever before, and political differences are widened and inflamed. They also provide new opportunities for protest and resistance

Following Tradition

Following Tradition is an expansive examination of the history of tradition— one of the most common as well as most contested terms in English language usage —in Americans\u27 thinking and discourse about culture. Tradition in use becomes problematic because of its multiple meanings and its conceptual softness. As a term and a concept, it has been important in the development of all scholarly fields that study American culture. Folklore, history, American studies, anthropology, cultural studies, and others assign different value and meaning to tradition. It is a frequent point of reference in popular discourse concerning everything from politics to lifestyles to sports and entertainment. Politicians and social advocates appeal to it as prima facie evidence of the worth of their causes. Entertainment and other media mass produce it, or at least a facsimile of it. In a society that frequently seeks to reinvent itself, tradition as a cultural anchor to be reverenced or rejected is an essential, if elusive, concept. Simon Bronner\u27s wide net captures the historical, rhetorical, philosophical, and psychological dimensions of tradition. As he notes, he has written a book about an American tradition—arguing about it. His elucidation of those arguments makes fascinating and thoughtful reading. An essential text for folklorists, Following Tradition will be a valuable reference as well for historians and anthropologists; students of American studies, popular culture, and cultural studies; and anyone interested in the continuing place of tradition in American culture.https://digitalcommons.usu.edu/usupress_pubs/1064/thumbnail.jp

Following Tradition

Following Tradition is an expansive examination of the history of traditio

Handbook of Well-Being

It is a pleasure to bring to you the eHandbook of Subjective Well-Being, the science of when and why people experience and evaluate their lives in positive ways, including aspects such as positive feelings, life satisfaction, and optimism. There are chapters in this eHandbook on the philosophy and history of well-being, as well as reviews of empirical research on the ways to assess well-being, the circumstances that predict it, the outcomes that it produces, the societal policies that enhance it, and many other social, biological, and cultural processes that help us understand why some people are happy and satisfied with their lives, while others are not. There are also chapters on theories of well-being, such as the baseline or set-point models. We believe that Open publication is the wave of the future (Jhangiani & Biswas-Diener, 2017). Therefore, we are presenting the handbook in an electronic format so that it is widely available to everyone around the world. The handbook is entirely open and free – anyone can read and use it without cost. This is important to us as we desire to lower knowledge barriers for individuals and communities, especially because it provides access to students, educators, and scholars who do not have substantial financial resources. We are not certain if this is the first free and open handbook in the behavioral sciences, but hopefully it will not be the last. In the past the prohibitive price of many handbooks have made them available only to scholars or institutions in wealthy nations, and this is unfortunate. We believe scientific scholarship should be available to all. The field of subjective well-being has grown at rapid pace over the last several decades, and many discoveries have been made. When Ed Diener began his research within the field in 1981 there were about 130 studies published that year on the topic, as shown using a Google Scholar search on “subjective well-being.” Eighteen years later when Shigehiro Oishi earned his Ph.D. in 2000 there were 1,640 publications that year on the topic, and when Louis Tay was awarded a Ph.D. in 2011 there were 10,400 publications about subjective well-being. Finally, in 2016 there were 18,300 publications – in that single year alone! In other words, during the time that Diener has been studying the topic, scholarship on subjective well-being has grown over 100-fold! It is not merely the number of published studies that has grown, but there have been enormous leaps forward in our understanding. In the 1980s, there were questions about the reliability and validity of subjective well-being assessments, and the components that underlie it. One notable advance is our understanding and measurement of well-being. We now know a great deal about the validity of self-report measures, as well as the core evaluative and affective components that make up subjective well-being. Further, scholars have a much greater understanding of the processes by which people report their subjective well-being, and various biases or artifacts that may influence these reports. In 1982 many studies were focused on demographic factors such as income, sex, and age that were correlated with subjective well-being. By 2016 we understood much more about temperament and other internal factors that influence happiness, as well as some of the outcomes in behavior that subjective well-being helps produce (e.g., income, performance, physical health, longevity). In the 1980s, researchers assumed that people adapt to almost any life event, and that different life events only have a short-term effect on subjective well-being. A number of large-scale longitudinal studies later showed that that is not the case. By now we know what kinds of life events affect our subjective well-being, how much, and for roughly how long. In the 1980s researchers believed that economic growth would not increase the happiness of a given nation. Now we know when economic growth tends to increase the happiness of a given nation. Additionally, we know much more about the biology of subjective well-being, and an enormous amount more about culture and well-being, a field that was almost nonexistent in 1982. With the advent of positive psychology, we are also beginning to examine practices and interventions that can raise subjective well-being. Given the broad interest in subjective well-being in multiple fields like psychology, economics, political science, and sociology, there have been important developments made toward understanding how societies differ in well-being. This understanding led to the development of national accounts of well-being – societies using well-being measures to help inform policy deliberations. This advance changes the focus of governments away from a narrow emphasis on economic development to a broader view which sees government policies as designed to raise human well-being. We were fortunate to have so many leading scholars of subjective well-being and related topics contribute to this volume. We might be slightly biased but most of the chapters in this eHandbook are truly superb. Not only do they provide a broad coverage of a large number of areas, but many of the chapters present new ways of thinking about these areas. Below is a brief overview of each of the sections in this volume: In Section 1 we begin the volume with chapters on philosophical, historical, and religious thinking on well-being through the ages. Next, we cover the methods and measures used in the scientific study of well-being. Section 2 is devoted to theories of well-being such as the top-down theory, activity theory, goal theory, self-determination theory, and evolutionary theory. Section 3 covers the personality, genetics, hormones, and neuroscience of well-being. Then, demographic factors such as age, gender, race, religion, and marital status are discussed. Section 4 is devoted to how domains of life – such as work, finance, close relationships, and leisure – are related to overall subjective well-being. Section 5 covers the various outcomes of subjective well-being, ranging from work outcomes, to cognitive outcomes, to health, and finally relationship outcomes. Section 6 covers interventions to increase subjective well-being. Finally, Section 7 is devoted to cultural, geographical, and historical variations in subjective well-being. This eHandbook presents the most up-to-date and comprehensive understanding of subjective well-being – and it is freely available to all! The editors would like to extend their thanks to several individuals who have been critical to the success of the handbook. First, our gratitude is immense toward Chris Wiese, Keya Biswas-Diener, and Danielle Geerling, who organized and kept the entire venture on track. Their hard work and organizational skills were wonderful, and the book would not have been possible without them. Second, we extend our thanks to the Diener Education Fund, a charitable organization devoted to education that in part made this project possible. In particular we express deep gratitude to Mary Alice and Frank Diener. Not only did their help make this eHandbook possible, but their lives stood as shining examples of the way to pursue well-being!https://digitalcommons.unomaha.edu/psychfacbooks/1008/thumbnail.jp