Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion

Towards Informed Exploration for Deep Reinforcement Learning

In this thesis, we discuss various techniques for improving exploration for deep reinforcement learning. We begin with a brief review of reinforcement learning (RL) and the fundamental v.s. exploitation trade-off. Then we review how deep RL has improved upon classical and summarize six categories of the latest exploration methods for deep RL, in the order increasing usage of prior information. We then explore representative works in three categories discuss their strengths and weaknesses. The first category, represented by Soft Q-learning, uses regularization to encourage exploration. The second category, represented by count-based via hashing, maps states to hash codes for counting and assigns higher exploration to less-encountered states. The third category utilizes hierarchy and is represented by modular architecture for RL agents to play StarCraft II. Finally, we conclude that exploration by prior knowledge is a promising research direction and suggest topics of potentially impact

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds.

We seek to create agents that both act and communicate with other agents in pursuit of a goal. Towards this end, we extend LIGHT (Urbanek et al. 2019)—a large-scale crowd-sourced fantasy text-game—with a dataset of quests. These contain natural language motivations paired with in-game goals and human demonstrations; completing a quest might require dialogue or actions (or both). We introduce a reinforcement learning system that (1) incorporates large-scale language modeling-based and commonsense reasoning-based pre-training to imbue the agent with relevant priors; and (2) leverages a factorized action space of action commands and dialogue, balancing between the two. We conduct zero-shot evaluations using held-out human expert demonstrations, showing that our agents are able to act consistently and talk naturally with respect to their motivations

Online Learning in Iterated Prisoner's Dilemma to Mimic Human Behavior

Prisoner's Dilemma mainly treat the choice to cooperate or defect as an atomic action. We propose to study online learning algorithm behavior in the Iterated Prisoner's Dilemma (IPD) game, where we explored the full spectrum of reinforcement learning agents: multi-armed bandits, contextual bandits and reinforcement learning. We have evaluate them based on a tournament of iterated prisoner's dilemma where multiple agents can compete in a sequential fashion. This allows us to analyze the dynamics of policies learned by multiple self-interested independent reward-driven agents, and also allows us study the capacity of these algorithms to fit the human behaviors. Results suggest that considering the current situation to make decision is the worst in this kind of social dilemma game. Multiples discoveries on online learning behaviors and clinical validations are stated.Comment: To the best of our knowledge, this is the first attempt to explore the full spectrum of reinforcement learning agents (multi-armed bandits, contextual bandits and reinforcement learning) in the sequential social dilemma. This mental variants section supersedes and extends our work arXiv:1706.02897 (MAB), arXiv:2005.04544 (CB) and arXiv:1906.11286 (RL) into the multi-agent settin

3D Sensing Character Simulation using Game Engine Physics

Creating visual 3D sensing characters that interact with AI peers and the virtual envi- ronment can be a difficult task for those with less experience in using learning algorithms or creating visual environments to execute an agent-based simulation. In this thesis, the use of game engines was studied as a tool to create and execute vi- sual simulations with 3D sensing characters, and train game ready bots. The idea was to make use of the game engine’s available tools to create highly visual simulations without requiring much knowledge in modeling or animation, as well as integrating exterior agent simulation libraries to create sensing characters without needing expertise in learning algorithms. These sensing characters, were be 3D humanoid characters that can perform the basic functions of a game character such as moving, jumping, and interacting, but also have simulated different senses in them. The senses that these characters can have include: touch using collision detection, vision using ray casts, directional sound, smell, and other imaginable senses. These senses are obtained using different game develop- ment techniques available in the game engine and can be used as input for the learning algorithm to help the character learn. This allows the simulation of agents using off-the- shelf algorithms and using the game engine’s motor for the visualizations of these agents. We explored the use of these tools to create visual bots for games, and teach them how to play the game until they reach a level where they can serve as adversaries for real-life players in interactive games. This solution was tested using both reinforcement learning and imitation learning algorithms in an attempt to compare how efficient both learning methods can be when used to teach sensing game bots in different game scenarios. These scenarios varied in both objective and environment complexity as well as the number of bots to access how each solution behaves in different scenarios. In this document is presented a related work on the agent simulation and game engine areas, followed by a more detailed solution and its implementation ending with practical tests and its results.Criar visualizações de personagens 3D com sentidos que interagem com colegas de IA e com o ambiente virtual pode ser uma tarefa difícil para programadores com menos experiência no uso de algoritmos de aprendizagem automática ou na criação de ambientes visuais para executar simulações baseadas em agentes. Nesta tese foi estudado o uso de motores de jogos como ferramenta para criar e execu- tar simulações visuais com personagens 3D, e treinar bots para jogos. A ideia foi usar as ferramentas disponíveis do motor de jogos para criar simulações visuais sem exigir muito conhecimento em modelação ou animação, para além de integrar bibliotecas de simulação de agentes externas para criar personagens com sentidos sem precisar de conhecimentos em algoritmos de aprendizagem automática. Estas personagens 3D são humanoides que podem desempenhar as funções básicas de uma personagem de um jogo como mover, saltar e interagir, mas também terão simulados neles diferentes sentidos. Os sentidos que estas personagens podem ter inclui: o tato, colisões, visão, som direcional, olfato e outros sentidos imagináveis. Estes sentidos são obtidos usando diferentes técnicas de desenvol- vimento de jogos disponíveis no motor de jogos, e podem ser usados como inputs para os algoritmos de aprendizagem automática para ajudar as personagens a aprender. Esta solução foi testada usando algoritmos de Reinforcement Learning e Imitation Le- arning, com o intuito de comparar a eficiência de ambos os métodos de aprendizagem quando usados para ensinar bots de jogos em diferentes cenários. Estes cenários variaram em complexidade de objetivo e ambiente, e também no número de bots para que se possa visualizar como cada algoritmo se comporta em diferentes cenários. Neste documento será apresentado um estado da arte nas áreas de simulação de agentes e motores de jogos, seguido de uma proposta de solução mais detalhada para este problema

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds

We seek to create agents that both act and communicate with other agents in pursuit of a goal. Towards this end, we extend LIGHT (Urbanek et al. 2019)---a large-scale crowd-sourced fantasy text-game---with a dataset of quests. These contain natural language motivations paired with in-game goals and human demonstrations; completing a quest might require dialogue or actions (or both). We introduce a reinforcement learning system that (1) incorporates large-scale language modeling-based and commonsense reasoning-based pre-training to imbue the agent with relevant priors; and (2) leverages a factorized action space of action commands and dialogue, balancing between the two. We conduct zero-shot evaluations using held-out human expert demonstrations, showing that our agents are able to act consistently and talk naturally with respect to their motivations

Demonstrating Advantages of Neuromorphic Computation: A Pilot Study

Neuromorphic devices represent an attempt to mimic aspects of the brain's architecture and dynamics with the aim of replicating its hallmark functional capabilities in terms of computational power, robust learning and energy efficiency. We employ a single-chip prototype of the BrainScaleS 2 neuromorphic system to implement a proof-of-concept demonstration of reward-modulated spike-timing-dependent plasticity in a spiking network that learns to play the Pong video game by smooth pursuit. This system combines an electronic mixed-signal substrate for emulating neuron and synapse dynamics with an embedded digital processor for on-chip learning, which in this work also serves to simulate the virtual environment and learning agent. The analog emulation of neuronal membrane dynamics enables a 1000-fold acceleration with respect to biological real-time, with the entire chip operating on a power budget of 57mW. Compared to an equivalent simulation using state-of-the-art software, the on-chip emulation is at least one order of magnitude faster and three orders of magnitude more energy-efficient. We demonstrate how on-chip learning can mitigate the effects of fixed-pattern noise, which is unavoidable in analog substrates, while making use of temporal variability for action exploration. Learning compensates imperfections of the physical substrate, as manifested in neuronal parameter variability, by adapting synaptic weights to match respective excitability of individual neurons.Comment: Added measurements with noise in NEST simulation, add notice about journal publication. Frontiers in Neuromorphic Engineering (2019