28,646 research outputs found
Hashing over Predicted Future Frames for Informed Exploration of Deep Reinforcement Learning
In deep reinforcement learning (RL) tasks, an efficient exploration mechanism
should be able to encourage an agent to take actions that lead to less frequent
states which may yield higher accumulative future return. However, both knowing
about the future and evaluating the frequentness of states are non-trivial
tasks, especially for deep RL domains, where a state is represented by
high-dimensional image frames. In this paper, we propose a novel informed
exploration framework for deep RL, where we build the capability for an RL
agent to predict over the future transitions and evaluate the frequentness for
the predicted future frames in a meaningful manner. To this end, we train a
deep prediction model to predict future frames given a state-action pair, and a
convolutional autoencoder model to hash over the seen frames. In addition, to
utilize the counts derived from the seen frames to evaluate the frequentness
for the predicted frames, we tackle the challenge of matching the predicted
future frames and their corresponding seen frames at the latent feature level.
In this way, we derive a reliable metric for evaluating the novelty of the
future direction pointed by each action, and hence inform the agent to explore
the least frequent one
Deep Reinforcement Learning for Event-Triggered Control
Event-triggered control (ETC) methods can achieve high-performance control
with a significantly lower number of samples compared to usual, time-triggered
methods. These frameworks are often based on a mathematical model of the system
and specific designs of controller and event trigger. In this paper, we show
how deep reinforcement learning (DRL) algorithms can be leveraged to
simultaneously learn control and communication behavior from scratch, and
present a DRL approach that is particularly suitable for ETC. To our knowledge,
this is the first work to apply DRL to ETC. We validate the approach on
multiple control tasks and compare it to model-based event-triggering
frameworks. In particular, we demonstrate that it can, other than many
model-based ETC designs, be straightforwardly applied to nonlinear systems
Crossmodal Attentive Skill Learner
This paper presents the Crossmodal Attentive Skill Learner (CASL), integrated
with the recently-introduced Asynchronous Advantage Option-Critic (A2OC)
architecture [Harb et al., 2017] to enable hierarchical reinforcement learning
across multiple sensory inputs. We provide concrete examples where the approach
not only improves performance in a single task, but accelerates transfer to new
tasks. We demonstrate the attention mechanism anticipates and identifies useful
latent features, while filtering irrelevant sensor modalities during execution.
We modify the Arcade Learning Environment [Bellemare et al., 2013] to support
audio queries, and conduct evaluations of crossmodal learning in the Atari 2600
game Amidar. Finally, building on the recent work of Babaeizadeh et al. [2017],
we open-source a fast hybrid CPU-GPU implementation of CASL.Comment: International Conference on Autonomous Agents and Multiagent Systems
(AAMAS) 2018, NIPS 2017 Deep Reinforcement Learning Symposiu
- …