Aprendizaje por refuerzo en sistemas multiagente mediante MARLÖ

[ES] En este trabajo de fin de grado se realizará un estudio basado en el análisis de la aplicación de algoritmos de aprendizaje por refuerzo para entornos mono-agente sobre entornos multi-agente basados en la plataforma MARLÖ. Todo esto con el objetivo de comparar la eficacia y eficiencia de dichos algoritmos en entornos para los cuales no han sido diseñados. Para esto será necesario tanto el diseño y creación de entornos personalizados como la modificación de las implementaciones de los algoritmos para adaptarlas a dichos entornos.[EN] In this final degree thesis, we perform a study based on the analysis of the application of reinforcement learning algorithms designed for single-agent environments on multi-agent environments based on the MARLÖ platform. All this in order to compare the effectiveness and efficiency of these algorithms in environments for which they have not been designed. This will require both the design and creation of custom environments and the modification of the algorithm implementations to adapt them to those environments.[CA] En aquest treball de fi de grau es realitzarà un estudi basat en l’anàlisi de l’aplicació d’algoritmes d’aprenentatge per reforç per a entorns mono-agent sobre entorns multiagent basats en la plataforma MARLÖ. Tot això amb l’objectiu de comparar l’eficàcia i eficiència d’aquests algorismes en entorns per als quals no han estat dissenyats. Per això serà necessari tant el disseny i creació d’entorns personalitzats com la modificació de les implementacions dels algoritmes per adaptar-les a aquests entorns.Gracias al Instituto Valenciano de Investigación en Inteligencia Artificial (VRAIN) por concederme la beca de formación que ha hecho este trabajo posible. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan V GPU used for this research.Martínez Sanchis, G. (2020). Aprendizaje por refuerzo en sistemas multiagente mediante MARLÖ. Universitat Politècnica de València. http://hdl.handle.net/10251/162288TFG

Beyond Playing to Win: Elicit General Gameplaying Agents with Distinct Behaviours to Assist Game Development and Testing

General Video Game Playing (GVGP) creates agents capable of playing several different games while maintaining competitive performance. Even when the generality of these agents has evident potential, there is a lack of research looking for applications for them. This work explores filling that void by advocating the integration of GVGP agents into the game development process. Additionally, it proposes studying the GVGP agents from a Player Experience perspective to facilitate their use in games as an alternative AI approach. GVGP agents are essentially designed to win and achieve a high score. However, the players' actions are driven by different motivations, resulting in diverse behaviours. These motivations may ultimately involve winning, but it is not necessarily their primary goal. Thus, why are agents designed with merely this purpose in mind? This work considers that the path that eventually allows finding applications for the agents starts with eliciting differentiated behaviours by providing them with objectives beyond winning. It introduces the concept of heuristic diversification that, in the scope of search algorithms, refers to isolating the evaluation function of the controllers providing the goals externally without affecting their foundation. This work proposes that a team of GVGP agents with differentiated behaviours can assist in the game development and testing processes. The solution applies heuristic diversification and describes the behaviour of an agent with simplicity and easiness to evolve. Diverse behaviours can be generated and used to assemble the team independently of the game's characteristics. Based on their stats, the resulting agents are allocated in a behavioural space, which is used to identify behaviour-type agents. The agents are portable between levels and facilitate diverse automated gameplay. They can detect design flaws and bugs when introducing modifications to the game or trigger external development tools without having to play the game manually

Intrinsic Motivation in Computational Creativity Applied to Videogames

PhD thesisComputational creativity (CC) seeks to endow artificial systems with creativity. Although human creativity is known to be substantially driven by intrinsic motivation (IM), most CC systems are extrinsically motivated. This restricts their actual and perceived creativity and autonomy, and consequently their benefit to people. In this thesis, we demonstrate, via theoretical arguments and through applications in videogame AI, that computational intrinsic reward and models of IM can advance core CC goals. We introduce a definition of IM to contextualise related work. Via two systematic reviews, we develop typologies of the benefits and applications of intrinsic reward and IM models in CC and game AI. Our reviews highlight that related work is limited to few reward types and motivations, and we thus investigate the usage of empowerment, a little studied, information-theoretic intrinsic reward, in two novel models applied to game AI. We define coupled empowerment maximisation (CEM), a social IM model, to enable general co-creative agents that support or challenge their partner through emergent behaviours. Via two qualitative, observational vignette studies on a custom-made videogame, we explore CEM’s ability to drive general and believable companion and adversary non-player characters which respond creatively to changes in their abilities and the game world. We moreover propose to leverage intrinsic reward to estimate people’s experience of interactive artefacts in an autonomous fashion. We instantiate this proposal in empowerment-based player experience prediction (EBPXP) and apply it to videogame procedural content generation. By analysing think-aloud data from an experiential vignette study on a dedicated game, we identify several experiences that EBPXP could predict. Our typologies serve as inspiration and reference for CC and game AI researchers to harness the benefits of IM in their work. Our new models can increase the generality, autonomy and creativity of next-generation videogame AI, and of CC systems in other domains