1,020 research outputs found
Distributional Reinforcement Learning with Quantile Regression
In reinforcement learning an agent interacts with the environment by taking
actions and observing the next state and reward. When sampled
probabilistically, these state transitions, rewards, and actions can all induce
randomness in the observed long-term return. Traditionally, reinforcement
learning algorithms average over this randomness to estimate the value
function. In this paper, we build on recent work advocating a distributional
approach to reinforcement learning in which the distribution over returns is
modeled explicitly instead of only estimating the mean. That is, we examine
methods of learning the value distribution instead of the value function. We
give results that close a number of gaps between the theoretical and
algorithmic results given by Bellemare, Dabney, and Munos (2017). First, we
extend existing results to the approximate distribution setting. Second, we
present a novel distributional reinforcement learning algorithm consistent with
our theoretical formulation. Finally, we evaluate this new algorithm on the
Atari 2600 games, observing that it significantly outperforms many of the
recent improvements on DQN, including the related distributional algorithm C51
IGN : Implicit Generative Networks
In this work, we build recent advances in distributional reinforcement
learning to give a state-of-art distributional variant of the model based on
the IQN. We achieve this by using the GAN model's generator and discriminator
function with the quantile regression to approximate the full quantile value
for the state-action return distribution. We demonstrate improved performance
on our baseline dataset - 57 Atari 2600 games in the ALE. Also, we use our
algorithm to show the state-of-art training performance of risk-sensitive
policies in Atari games with the policy optimization and evaluation
GAN-powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing
Network slicing is a key technology in 5G communications system. Its purpose
is to dynamically and efficiently allocate resources for diversified services
with distinct requirements over a common underlying physical infrastructure.
Therein, demand-aware resource allocation is of significant importance to
network slicing. In this paper, we consider a scenario that contains several
slices in a radio access network with base stations that share the same
physical resources (e.g., bandwidth or slots). We leverage deep reinforcement
learning (DRL) to solve this problem by considering the varying service demands
as the environment state and the allocated resources as the environment action.
In order to reduce the effects of the annoying randomness and noise embedded in
the received service level agreement (SLA) satisfaction ratio (SSR) and
spectrum efficiency (SE), we primarily propose generative adversarial
network-powered deep distributional Q network (GAN-DDQN) to learn the
action-value distribution driven by minimizing the discrepancy between the
estimated action-value distribution and the target action-value distribution.
We put forward a reward-clipping mechanism to stabilize GAN-DDQN training
against the effects of widely-spanning utility values. Moreover, we further
develop Dueling GAN-DDQN, which uses a specially designed dueling generator, to
learn the action-value distribution by estimating the state-value distribution
and the action advantage function. Finally, we verify the performance of the
proposed GAN-DDQN and Dueling GAN-DDQN algorithms through extensive
simulations
- …