3,808 research outputs found
GAN-powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing
Network slicing is a key technology in 5G communications system. Its purpose
is to dynamically and efficiently allocate resources for diversified services
with distinct requirements over a common underlying physical infrastructure.
Therein, demand-aware resource allocation is of significant importance to
network slicing. In this paper, we consider a scenario that contains several
slices in a radio access network with base stations that share the same
physical resources (e.g., bandwidth or slots). We leverage deep reinforcement
learning (DRL) to solve this problem by considering the varying service demands
as the environment state and the allocated resources as the environment action.
In order to reduce the effects of the annoying randomness and noise embedded in
the received service level agreement (SLA) satisfaction ratio (SSR) and
spectrum efficiency (SE), we primarily propose generative adversarial
network-powered deep distributional Q network (GAN-DDQN) to learn the
action-value distribution driven by minimizing the discrepancy between the
estimated action-value distribution and the target action-value distribution.
We put forward a reward-clipping mechanism to stabilize GAN-DDQN training
against the effects of widely-spanning utility values. Moreover, we further
develop Dueling GAN-DDQN, which uses a specially designed dueling generator, to
learn the action-value distribution by estimating the state-value distribution
and the action advantage function. Finally, we verify the performance of the
proposed GAN-DDQN and Dueling GAN-DDQN algorithms through extensive
simulations
Crop Yield Prediction Using Deep Neural Networks
Crop yield is a highly complex trait determined by multiple factors such as
genotype, environment, and their interactions. Accurate yield prediction
requires fundamental understanding of the functional relationship between yield
and these interactive factors, and to reveal such relationship requires both
comprehensive datasets and powerful algorithms. In the 2018 Syngenta Crop
Challenge, Syngenta released several large datasets that recorded the genotype
and yield performances of 2,267 maize hybrids planted in 2,247 locations
between 2008 and 2016 and asked participants to predict the yield performance
in 2017. As one of the winning teams, we designed a deep neural network (DNN)
approach that took advantage of state-of-the-art modeling and solution
techniques. Our model was found to have a superior prediction accuracy, with a
root-mean-square-error (RMSE) being 12% of the average yield and 50% of the
standard deviation for the validation dataset using predicted weather data.
With perfect weather data, the RMSE would be reduced to 11% of the average
yield and 46% of the standard deviation. We also performed feature selection
based on the trained DNN model, which successfully decreased the dimension of
the input space without significant drop in the prediction accuracy. Our
computational results suggested that this model significantly outperformed
other popular methods such as Lasso, shallow neural networks (SNN), and
regression tree (RT). The results also revealed that environmental factors had
a greater effect on the crop yield than genotype.Comment: 9 pages, Presented at 2018 INFORMS Conference on Business Analytics
and Operations Research (Baltimore, MD, USA). One of the winning solutions to
the 2018 Syngenta Crop Challeng
A Comparative Analysis of Expected and Distributional Reinforcement Learning
Since their introduction a year ago, distributional approaches to
reinforcement learning (distributional RL) have produced strong results
relative to the standard approach which models expected values (expected RL).
However, aside from convergence guarantees, there have been few theoretical
results investigating the reasons behind the improvements distributional RL
provides. In this paper we begin the investigation into this fundamental
question by analyzing the differences in the tabular, linear approximation, and
non-linear approximation settings. We prove that in many realizations of the
tabular and linear approximation settings, distributional RL behaves exactly
the same as expected RL. In cases where the two methods behave differently,
distributional RL can in fact hurt performance when it does not induce
identical behaviour. We then continue with an empirical analysis comparing
distributional and expected RL methods in control settings with non-linear
approximators to tease apart where the improvements from distributional RL
methods are coming from.Comment: To appear in the Proceedings of the Thirty-Third AAAI Conference on
Artificial Intelligenc
- …