Teaching Analog Skills in a Digital World

This paper explores the issue of the socio-cultural transformations implicating a change in the learning and teaching processes due to the extensive digitization of human environments. The purpose of the research is to deliver a contextualized insight into the effects of the digitalization of learning and other parts of the public sphere. This issue is being investigated through such notions as nostalgia, retromania, and retro learning with the methods of discourse analysis and elements of ethnography of organizations. It is thus important to point out the most important cultural tendencies in the dynamics of this change by bringing out the significance of non-digital learning patterns. The research’s outcome highlights these patterns which build so-called analog cultures, i.e. specific forms of identity construction in the digital setting which emphasizes the importance of the relations between man and machine in the context of skills acquisition for example. The takeaway from the research might be implemented in fields like organizational studies and schooling reforms aimed at improving the effectiveness of digital learnin

Deeplogger: Extracting user input logs from 2D gameplay videos

Game and player analysis would be much easier if user interactions were electronically logged and shared with game researchers. Understandably, sniffing software is perceived as invasive and a risk to privacy. To collect player analytics from large populations, we look to the millions of users who already publicly share video of their game playing. Though labor-intensive, we found that someone with experience of playing a specific game can watch a screen-cast of someone else playing, and can then infer approximately what buttons and controls the player pressed, and when. We seek to automatically convert video into such game-play transcripts, or logs. We approach the task of inferring user interaction logs from video as a machine learning challenge. Specifically, we propose a supervised learning framework to first train a neural network on videos, where real sniffer/instrumented software was collecting ground truth logs. Then, once our DeepLogger network is trained, it should ideally infer log-activities for each new input video, which features gameplay of that game. These user-interaction logs can serve as sensor data for gaming analytics, or as supervision for training of game-playing AI’s. We evaluate the DeepLogger system for generating logs from two 2D games, Tetris [23] and Mega Man X [6], chosen to represent distinct game genres. Our system performs as well as human experts for the task of video-to-log transcription, and could allow game researchers to easily scale their data collection and analysis up to massive populations

TEACHING ANALOG SKILLS IN A DIGITAL WORLD

This paper explores the issue of the socio-cultural transformations implicating a change in the learning and teaching processes due to the extensive digitization of human environments. The purpose of the research is to deliver a contextualized insight into the effects of the digitalization of learning and other parts of the public sphere. This issue is being investigated through such notions as nostalgia, retromania, and retro learning with the methods of discourse analysis and elements of ethnography of organizations. It is thus important to point out the most important cultural tendencies in the dynamics of this change by bringing out the significance of non-digital learning patterns. The research’s outcome highlights these patterns which build so-called analog cultures, i.e. specific forms of identity construction in the digital setting which emphasizes the importance of the relations between man and machine in the context of skills acquisition for example. The takeaway from the research might be implemented in fields like organizational studies and schooling reforms aimed at improving the effectiveness of digital learning

Aprendiendo de la memoria RAM de la NES

En este proyecto se experimentó con algoritmos de aprendizaje por refuerzo jugando Donkey Kong, Ice Climber, Kung Fu, Super Mario Bros y Metroid de la consola NES. El algoritmo DQN y sus variantes como Doble DQN y Dueling DQN fueron usados para experimentar con la representación de estados mediante la RAM. Se proponen algunas estrategias para reducir la dimensionalidad de los estados y las acciones. También se proponen funciones de recompensa para crear un agente fácil de entrenar con pocos recursos computacionales. Se probaron dos maneras de reducir la dimensión de la RAM, el mapa de la RAM funcionó bien sólo en fase de entrenamiento mientras que el método de los bytes activados consiguió mejores resultados.In this project, reinforcement learning algorithms are explored to play the games Donkey Kong, Ice Climber, Kung Fu, Super Mario Bros, and Metroid from the NES console. The Deep-Q learning algorithm is used to experiment with the RAM representation of the state. DQN algorithm extensions as Double DQN and Dueling Network Architectures are explored too. Some simple strategies to reduce the state-space and action-space are proposed in addition to reward functions to create an easy to train agent with low computational resources. Two ways to reduce the RAM representation of the state were tested, RAM map method worked well just in the training phase meanwhile the activated bytes method get better results also in gameplay.En aquest projecte es va experimentar amb algorismes d'aprenentatge per reforç jugant Donkey Kong, Hissi Climber, Kung Fu, Super Mario Bros i Metroid de la consola NES. L'algorisme DQN i les seves variants com a Doble DQN i Dueling DQN van ser usats per a experimentar amb la representació d'estats mitjançant la RAM. Es proposen algunes estratègies per a reduir la dimensionalitat dels estats i les accions. També es proposen funcions de recompensa per a crear un agent fàcil d'entrenar amb pocs recursos computacionals. Es van provar dues maneres de reduir la dimensió de la RAM, el mapa de la RAM va funcionar bé només en fase d'entrenament mentre que el mètode dels bytes activats va aconseguir millors resultats

Reinforcement Learning for Generative Art

Reinforcement learning (RL) is an efficient class of sequential decision-making algorithms that have achieved remarkable success in a broad range of applications, such as robotic manipulations, strategic games, or autonomous driving. The most well-known example of reinforcement learning is AlphaGo, a computer program that plays the board game Go and outperforms top human Go players. Unlike other two major machine learning categories, supervised learning and unsupervised learning, in which media artists are actively engaged, reinforcement learning has yet to result in many creative applications. Generative art is usually driven, in whole or in part, by autonomous systems that are derived from a set of rules. Interestingly, an RL policy can be seen as an autonomous system where the rules are learned by interacting with its environment. Regardless of its initial purpose, reinforcement learning has the potential to expand the boundary of generative art. However, a formal process of applying reinforcement learning to generative art does not yet exist and the current RL tools require an in-depth understanding of RL concepts. To bridge the gap, the first part of the dissertation introduces a conceptual framework to adapt reinforcement learning for generative art. The framework proposes a term RL-based generative art to denote a novel form of generative art of which the use of RL agents is the key element. The creative process of RL-based generative art and possible emergent behaviors are discussed in the framework. This leads to a discussion of several author's related practices on generative art, deep-learning art, and reinforcement learning. Those practices are critical for understanding the conceptual and technical details of each component in order to construct the framework. The second part introduces RL5, a JavaScript library for rapidly prototyping RL environments and training RL policies in web browsers. The library combines RL algorithms and RL environments into one framework and is fully compatible with p5.js. RL5 is developed with a particular focus on simplicity to favor (re)usability of RL algorithms and development of RL environments. Specifically, the library implemented three RL algorithms, Tabular Q-learning, REINFORCE, and DDPG, to cover all the three families of model-free RL, and nine RL environments that six of them address autonomous agents in steering behaviors, which can be used as building blocks for complex systems. Finally, the author demonstrates four different use cases of how to apply RL5 for pedagogical and creative applications

Learn to automate GUI tasks from demonstration

This thesis explores and extends Computer Vision applications in the context of Graphical User Interface (GUI) environments to address the challenges of Programming by Demonstration (PbD). The challenges are explored in PbD which could be addressed through innovations in Computer Vision, when GUIs are treated as an application domain, analogous to automotive or factory settings. Existing PbD systems were restricted by domain applications or special application interfaces. Although they use the term Demonstration, the systems did not actually see what the user performs. Rather they listen to the demonstrations through internal communications via operating system. Machine Vision and Human in the Loop Machine Learning are used to circumvent many restrictions, allowing the PbD system to watch the demonstration like another human observer would. This thesis will demonstrate that our prototype PbD systems allow non-programmer users to easily create their own automation scripts for their repetitive and looping tasks. Our PbD systems take their input from sequences of screenshots, and sometimes from easily available keyboard and mouse sniffer software. It will also be shown that the problem of inconsistent human demonstration can be remedied with our proposed Human in the Loop Computer Vision techniques. Lastly, the problem is extended to learn from demonstration videos. Due to the sheer complexity of computer desktop GUI manipulation videos, attention is focused on the domain of video game environments. The initial studies illustrate that it is possible to teach a computer to watch gameplay videos and to estimate what buttons the user pressed