114 research outputs found
GAN-powered Deep Distributional Reinforcement Learning for Resource Management in Network Slicing
Network slicing is a key technology in 5G communications system. Its purpose
is to dynamically and efficiently allocate resources for diversified services
with distinct requirements over a common underlying physical infrastructure.
Therein, demand-aware resource allocation is of significant importance to
network slicing. In this paper, we consider a scenario that contains several
slices in a radio access network with base stations that share the same
physical resources (e.g., bandwidth or slots). We leverage deep reinforcement
learning (DRL) to solve this problem by considering the varying service demands
as the environment state and the allocated resources as the environment action.
In order to reduce the effects of the annoying randomness and noise embedded in
the received service level agreement (SLA) satisfaction ratio (SSR) and
spectrum efficiency (SE), we primarily propose generative adversarial
network-powered deep distributional Q network (GAN-DDQN) to learn the
action-value distribution driven by minimizing the discrepancy between the
estimated action-value distribution and the target action-value distribution.
We put forward a reward-clipping mechanism to stabilize GAN-DDQN training
against the effects of widely-spanning utility values. Moreover, we further
develop Dueling GAN-DDQN, which uses a specially designed dueling generator, to
learn the action-value distribution by estimating the state-value distribution
and the action advantage function. Finally, we verify the performance of the
proposed GAN-DDQN and Dueling GAN-DDQN algorithms through extensive
simulations
TACT: A Transfer Actor-Critic Learning Framework for Energy Saving in Cellular Radio Access Networks
Recent works have validated the possibility of improving energy efficiency in
radio access networks (RANs), achieved by dynamically turning on/off some base
stations (BSs). In this paper, we extend the research over BS switching
operations, which should match up with traffic load variations. Instead of
depending on the dynamic traffic loads which are still quite challenging to
precisely forecast, we firstly formulate the traffic variations as a Markov
decision process. Afterwards, in order to foresightedly minimize the energy
consumption of RANs, we design a reinforcement learning framework based BS
switching operation scheme. Furthermore, to avoid the underlying curse of
dimensionality in reinforcement learning, a transfer actor-critic algorithm
(TACT), which utilizes the transferred learning expertise in historical periods
or neighboring regions, is proposed and provably converges. In the end, we
evaluate our proposed scheme by extensive simulations under various practical
configurations and show that the proposed TACT algorithm contributes to a
performance jumpstart and demonstrates the feasibility of significant energy
efficiency improvement at the expense of tolerable delay performance.Comment: 11 figures, 30 pages, accepted in IEEE Transactions on Wireless
Communications 2014. IEEE Trans. Wireless Commun., Feb. 201
Two-tier Spatial Modeling of Base Stations in Cellular Networks
Poisson Point Process (PPP) has been widely adopted as an efficient model for
the spatial distribution of base stations (BSs) in cellular networks. However,
real BSs deployment are rarely completely random, due to environmental impact
on actual site planning. Particularly, for multi-tier heterogeneous cellular
networks, operators have to place different BSs according to local coverage and
capacity requirement, and the diversity of BSs' functions may result in
different spatial patterns on each networking tier. In this paper, we consider
a two-tier scenario that consists of macrocell and microcell BSs in cellular
networks. By analyzing these two tiers separately and applying both classical
statistics and network performance as evaluation metrics, we obtain accurate
spatial model of BSs deployment for each tier. Basically, we verify the
inaccuracy of using PPP in BS locations modeling for either macrocells or
microcells. Specifically, we find that the first tier with macrocell BSs is
dispersed and can be precisely modelled by Strauss point process, while Matern
cluster process captures the second tier's aggregation nature very well. These
statistical models coincide with the inherent properties of macrocell and
microcell BSs respectively, thus providing a new perspective in understanding
the relationship between spatial structure and operational functions of BSs
Traffic Prediction Based on Random Connectivity in Deep Learning with Long Short-Term Memory
Traffic prediction plays an important role in evaluating the performance of
telecommunication networks and attracts intense research interests. A
significant number of algorithms and models have been put forward to analyse
traffic data and make prediction. In the recent big data era, deep learning has
been exploited to mine the profound information hidden in the data. In
particular, Long Short-Term Memory (LSTM), one kind of Recurrent Neural Network
(RNN) schemes, has attracted a lot of attentions due to its capability of
processing the long-range dependency embedded in the sequential traffic data.
However, LSTM has considerable computational cost, which can not be tolerated
in tasks with stringent latency requirement. In this paper, we propose a deep
learning model based on LSTM, called Random Connectivity LSTM (RCLSTM).
Compared to the conventional LSTM, RCLSTM makes a notable breakthrough in the
formation of neural network, which is that the neurons are connected in a
stochastic manner rather than full connected. So, the RCLSTM, with certain
intrinsic sparsity, have many neural connections absent (distinguished from the
full connectivity) and which leads to the reduction of the parameters to be
trained and the computational cost. We apply the RCLSTM to predict traffic and
validate that the RCLSTM with even 35% neural connectivity still shows a
satisfactory performance. When we gradually add training samples, the
performance of RCLSTM becomes increasingly closer to the baseline LSTM.
Moreover, for the input traffic sequences of enough length, the RCLSTM exhibits
even superior prediction accuracy than the baseline LSTM.Comment: 6 pages, 9 figure
- …