Aprendizagem de coordenação em sistemas multi-agente

The ability for an agent to coordinate with others within a system is a valuable property in multi-agent systems. Agents either cooperate as a team to accomplish a common goal, or adapt to opponents to complete different goals without being exploited. Research has shown that learning multi-agent coordination is significantly more complex than learning policies in singleagent environments, and requires a variety of techniques to deal with the properties of a system where agents learn concurrently. This thesis aims to determine how can machine learning be used to achieve coordination within a multi-agent system. It asks what techniques can be used to tackle the increased complexity of such systems and their credit assignment challenges, how to achieve coordination, and how to use communication to improve the behavior of a team. Many algorithms for competitive environments are tabular-based, preventing their use with high-dimension or continuous state-spaces, and may be biased against specific equilibrium strategies. This thesis proposes multiple deep learning extensions for competitive environments, allowing algorithms to reach equilibrium strategies in complex and partially-observable environments, relying only on local information. A tabular algorithm is also extended with a new update rule that eliminates its bias against deterministic strategies. Current state-of-the-art approaches for cooperative environments rely on deep learning to handle the environment’s complexity and benefit from a centralized learning phase. Solutions that incorporate communication between agents often prevent agents from being executed in a distributed manner. This thesis proposes a multi-agent algorithm where agents learn communication protocols to compensate for local partial-observability, and remain independently executed. A centralized learning phase can incorporate additional environment information to increase the robustness and speed with which a team converges to successful policies. The algorithm outperforms current state-of-the-art approaches in a wide variety of multi-agent environments. A permutation invariant network architecture is also proposed to increase the scalability of the algorithm to large team sizes. Further research is needed to identify how can the techniques proposed in this thesis, for cooperative and competitive environments, be used in unison for mixed environments, and whether they are adequate for general artificial intelligence.A capacidade de um agente se coordenar com outros num sistema é uma propriedade valiosa em sistemas multi-agente. Agentes cooperam como uma equipa para cumprir um objetivo comum, ou adaptam-se aos oponentes de forma a completar objetivos egoístas sem serem explorados. Investigação demonstra que aprender coordenação multi-agente é significativamente mais complexo que aprender estratégias em ambientes com um único agente, e requer uma variedade de técnicas para lidar com um ambiente onde agentes aprendem simultaneamente. Esta tese procura determinar como aprendizagem automática pode ser usada para encontrar coordenação em sistemas multi-agente. O documento questiona que técnicas podem ser usadas para enfrentar a superior complexidade destes sistemas e o seu desafio de atribuição de crédito, como aprender coordenação, e como usar comunicação para melhorar o comportamento duma equipa. Múltiplos algoritmos para ambientes competitivos são tabulares, o que impede o seu uso com espaços de estado de alta-dimensão ou contínuos, e podem ter tendências contra estratégias de equilíbrio específicas. Esta tese propõe múltiplas extensões de aprendizagem profunda para ambientes competitivos, permitindo a algoritmos atingir estratégias de equilíbrio em ambientes complexos e parcialmente-observáveis, com base em apenas informação local. Um algoritmo tabular é também extendido com um novo critério de atualização que elimina a sua tendência contra estratégias determinísticas. Atuais soluções de estado-da-arte para ambientes cooperativos têm base em aprendizagem profunda para lidar com a complexidade do ambiente, e beneficiam duma fase de aprendizagem centralizada. Soluções que incorporam comunicação entre agentes frequentemente impedem os próprios de ser executados de forma distribuída. Esta tese propõe um algoritmo multi-agente onde os agentes aprendem protocolos de comunicação para compensarem por observabilidade parcial local, e continuam a ser executados de forma distribuída. Uma fase de aprendizagem centralizada pode incorporar informação adicional sobre ambiente para aumentar a robustez e velocidade com que uma equipa converge para estratégias bem-sucedidas. O algoritmo ultrapassa abordagens estado-da-arte atuais numa grande variedade de ambientes multi-agente. Uma arquitetura de rede invariante a permutações é também proposta para aumentar a escalabilidade do algoritmo para grandes equipas. Mais pesquisa é necessária para identificar como as técnicas propostas nesta tese, para ambientes cooperativos e competitivos, podem ser usadas em conjunto para ambientes mistos, e averiguar se são adequadas a inteligência artificial geral.Apoio financeiro da FCT e do FSE no âmbito do III Quadro Comunitário de ApoioPrograma Doutoral em Informátic

Reinforcement learning para videojuegos

En este proyecto, se hará uso de técnicas de aprendizaje por refuerzo para desarrollar un agente inteligente capaz de jugar batallas Pokémon en el servidor de Pokémon Showdown. Se hará una pequeña introducción acerca del tema, además de desarrollar agentes simples en entornos clásicos y de Atari, para después terminar creando el agente de Pokémon

The Adaptive Contexts of Videogame Adaptations and Franchises across Media

Videogame adaptations have been a staple of cinema and television since the 1980s and have had a consistent presence despite receiving overwhelmingly negative reactions. Recognising the perseverance of videogame adaptations, I examine some of the key issues and debates surrounding the genre with in-depth analysis of the source material, the machinations of the film and videogame industries, and the films themselves, specifically relating to three prominent onscreen videogame adaptations. Following an introduction to the various theories and areas of study already performed in this field, all of which I incorporate into an intricate, blended methodology, I explore issues of fidelity, localisation, and evolution that occur when adapting Sonic the Hedgehog out of the confines of its limited narrative. In examining adaptations of Mortal Kombat and Street Fighter, I explore how cinematic genres (such as the Hong Kong martial arts and American action movies) have influenced the creation of videogames and the production of their film and television adaptations. Finally, I delve into the history of zombie horror films, which influenced the Resident Evil franchise. As this became the longest-running (and, by extension, most successful) live-action videogame franchise, I explore the complex production of videogame adaptations, their critical and financial reception, and their ability to evolve into multimedia franchises. Overall, my work is designed to take videogame adaptations seriously by examining them through in-depth analysis, exploring how they convey the gameplay mechanics of their source material, analysing why they remain so popular despite their negative reputation, and by establishing an academic framework by which to discuss them with the same reverence afforded to literary adaptations

Social, Casual and Mobile Games

This book is available as open access through the Bloomsbury Open Access programme and is available on www.bloomsburycollections.com. Social, casual and mobile games, played on devices such as smartphones, tablets, or PCs and accessed through online social networks, have become extremely popular, and are changing the ways in which games are designed, understood, and played. These games have sparked a revolution as more people from a broader demographic than ever play games, shifting the stereotype of gaming away from that of hardcore, dedicated play to that of activities that fit into everyday life. Social, Casual and Mobile Games explores the rapidly changing gaming landscape and discusses the ludic, methodological, theoretical, economic, social and cultural challenges that these changes invoke. With chapters discussing locative games, the new freemium economic model, and gamer demographics, as well as close studies of specific games (including Candy Crush Saga, Angry Birds, and Ingress), this collection offers an insight into the changing nature of games and the impact that mobile media is having upon individuals and societies around the world

Strategic Latency Unleashed: The Role of Technology in a Revisionist Global Order and the Implications for Special Operations Forces

The article of record may be found at https://cgsr.llnl.govThis work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory in part under Contract W-7405-Eng-48 and in part under Contract DE-AC52-07NA27344. The views and opinions of the author expressed herein do not necessarily state or reflect those of the United States government or Lawrence Livermore National Security, LLC. ISBN-978-1-952565-07-6 LCCN-2021901137 LLNL-BOOK-818513 TID-59693This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory in part under Contract W-7405-Eng-48 and in part under Contract DE-AC52-07NA27344. The views and opinions of the author expressed herein do not necessarily state or reflect those of the United States government or Lawrence Livermore National Security, LLC. ISBN-978-1-952565-07-6 LCCN-2021901137 LLNL-BOOK-818513 TID-5969

Games with Words: Textual Representation in the Wake of Graphical Realism in Videogames

Much of the videogame industry is based around a model of technological progress, whereby developers, individual videogames, and videogame platforms are lauded as superior based on their engagement with the latest, cutting edge forms of technology. As a direct consequence of this focus, sophisticated graphic-based representations are often employed as a yardstick for technological superiority, as it is a form of advancement that can be discerned by the naked eye. The focus on graphics has a number of consequences: it presents past videogames as inferior realizations that pale before more modern approaches; it favors image-based representation over other representational forms such as text; it enters videogames into a broader, ongoing debate in Western culture regarding mimesis and representations of reality that pit image and text against each other. An alternative to the graphic-dominated history of videogames is a variantological approach, in which marginalized and past forms of representation are not seen as dead ends and failures, but as variants that offer alternative perspectives. To that end, this dissertation analyzes five different text-based variant approaches that present ways of considering videogames apart from the dominant narrative of technology-driven graphical realism. First, the history of the instruction manual illustrates how a text-based paratext functions in regards to videogames, which can be viewed as reinforcing the technology-driven approach to videogames—up to and including technology rendering the manual defunct—but can also illustrate a second history, one which explores how a manual as paratext acts to support videogames through incorporation of other print media forms such as the comic book or the picturebook, through presenting a model of the ideal gamer, and through presenting the manual itself as an object from within the videogame. Second, in the history of videogame technology, the 1980s are a crucial period, in which console systems developed an image-based vocabulary for nascent players to learn, and the text-based videogames of the personal computer looked to literary models to do what the image-based games could not; the 1989 Amiga game It Came From the Desert represents the moment when text and image cease to be competing forms and turn into formations more complementary. Third, in the 1990s, this balance shifts towards image with the advent of 3D immersive graphics, and the dominance of graphic-based realism, as illustrated through DOOM and Myst—though both games not only used text, but depended on it to engage players to enter into make-believe, mimetic games with their respective gameworlds. In the face of graphical realism’s dominance, the 1999 computer game Planescape: Torment stands out as a text-heavy variant, illustrating the ability of textual representation to engage with mimesis-as-make-believe and offer an alternative to graphical realism through self-inscription, the presentation of text, and a gameworld based on the power of belief and words. Finally, a fifth approach to textual variants comes through a consideration of the role of the text-centred artifact the book within videogames, which presents a wide variety of uses including the book as epitext, book as narrative frame, book as menu system, book as found object, and book as allusive structure

Exploring Game Transfer Phenomena: a multimodal research approach for investigating video games' effects

Video games are evolving and are becoming ever more immersive. Consequently, it is necessary to understand their effects on gamers’ psychological wellbeing. The impact on cognition, affect and behaviour has mostly been investigated separately and sometimes from narrow approaches that limit the understanding of the video games’ effects. This thesis investigates the effects of playing video games from a novel, multimodal and broad research approach that is termed "Game Transfer Phenomena" (GTP). The Game Transfer Phenomena framework examines the influence of video games on gamers’ sensorial perception, cognitions, and behaviours directly related to video games’ structural characteristic, game content and, in-game activities. A theoretically eclectic approach is taken to explain the interplay of physiological, perceptual, and cognitive mechanisms involved in GTP, mainly informed by socio-cognitive and behavioural theories. Mixed-methods were used in the empirical research. The three qualitative studies presented here were analysed by content and thematic analysis methodologies (n=1,244), and the quantitative online survey using appropriate statistical testing (n=2,362). The specific aims of the thesis were: (i) identify, classify and explain GTP experiences collected in online video game forums divided in three empirical studies, and (ii) investigate the characteristics of GTP, and factors associated with GTP in a cross-cultural online survey with a total sample of English and Spanish Speaking gamers. The results suggest that gamers experience a variety of non-volitional phenomena when not playing. (i) Experiencing thoughts, urges, images, sounds, tactile and kinaesthetic perceptions and sensations associated with the game, (ii) perceiving distorted physical stimuli due to the video game features, (iii) confusing physical stimuli with video game elements, (iv) responding to certain physical stimuli as in the game, and (v) experiencing involuntary body movements and behaviours directly related with the video game. In summary: (i) different gamers reported similar GTP in the same video games, (ii) GTP are in the continuum between normal and pathological phenomena, and appear to reflect failures in cognitive and control inhibition, and neural adaptations, (iii) age and occupation, having a medical condition, gamer type, session length, playing for escape, immersion, exploring, customization and rules and mechanics were significantly associated with GTP, and (iv) the appraisal of GTP were either positive or negative (with one in five players experiencing distress or dysfunction due to their GTP experiences). Findings suggest that resemblances between virtual and physical objects facilitate GTP and these may be strengthened with more advanced technologies. Knowing about particular video game features and their effect on gamers may contribute to taking more informed decisions regarding the psychological, cognitive, physiological and social effects of video games and the technologies that are still to come