365 research outputs found
Deep Q-Learning for Self-Organizing Networks Fault Management and Radio Performance Improvement
We propose an algorithm to automate fault management in an outdoor cellular
network using deep reinforcement learning (RL) against wireless impairments.
This algorithm enables the cellular network cluster to self-heal by allowing RL
to learn how to improve the downlink signal to interference plus noise ratio
through exploration and exploitation of various alarm corrective actions. The
main contributions of this paper are to 1) introduce a deep RL-based fault
handling algorithm which self-organizing networks can implement in a polynomial
runtime and 2) show that this fault management method can improve the radio
link performance in a realistic network setup. Simulation results show that our
proposed algorithm learns an action sequence to clear alarms and improve the
performance in the cellular cluster better than existing algorithms, even
against the randomness of the network fault occurrences and user movements.Comment: (c) 2018 IEEE. Personal use of this material is permitted. Permission
from IEEE must be obtained for all other uses, in any current or future
media, including reprinting/republishing this material for advertising or
promotional purposes, creating new collective works, for resale or
redistribution to servers or lists, or reuse of any copyrighted component of
this work in other work
Model Based Residual Policy Learning with Applications to Antenna Control
Non-differentiable controllers and rule-based policies are widely used for
controlling real systems such as telecommunication networks and robots.
Specifically, parameters of mobile network base station antennas can be
dynamically configured by these policies to improve users coverage and quality
of service. Motivated by the antenna tilt control problem, we introduce
Model-Based Residual Policy Learning (MBRPL), a practical reinforcement
learning (RL) method. MBRPL enhances existing policies through a model-based
approach, leading to improved sample efficiency and a decreased number of
interactions with the actual environment when compared to off-the-shelf RL
methods.To the best of our knowledge, this is the first paper that examines a
model-based approach for antenna control. Experimental results reveal that our
method delivers strong initial performance while improving sample efficiency
over previous RL methods, which is one step towards deploying these algorithms
in real networks
Cooperative Tri-Point Model-Based Ground-to-Air Coverage Extension in Beyond 5G Networks
The utilization of existing terrestrial infrastructures to provide coverage
for aerial users is a potentially low-cost solution. However, the already
deployed terrestrial base stations (TBSs) result in weak ground-to-air (G2A)
coverage due to the down-tilted antennas. Furthermore, achieving optimal
coverage across the entire airspace through antenna adjustment is challenging
due to the complex signal coverage requirements in three-dimensional space,
especially in the vertical direction. In this paper, we propose a cooperative
tri-point (CoTP) model-based method that utilizes cooperative beams to enhance
the G2A coverage extension. To utilize existing TBSs for establishing effective
cooperation, we prove that the cooperation among three TBSs can ensure G2A
coverage with a minimum coverage overlap, and design the CoTP model to analyze
the G2A coverage extension. Using the model, a cooperative coverage structure
based on Delaunay triangulation is designed to divide triangular prism-shaped
subspaces and corresponding TBS cooperation sets. To enable TBSs in the
cooperation set to cover different height subspaces while maintaining ground
coverage, we design a cooperative beam generation algorithm to maximize the
coverage in the triangular prism-shaped airspace. The simulation results and
field trials demonstrate that the proposed method can efficiently enhance the
G2A coverage extension while guaranteeing ground coverage
A survey of machine learning techniques applied to self organizing cellular networks
In this paper, a survey of the literature of the past fifteen years involving Machine Learning (ML) algorithms applied to self organizing cellular networks is performed. In order for future networks to overcome the current limitations and address the issues of current cellular systems, it is clear that more intelligence needs to be deployed, so that a fully autonomous and flexible network can be enabled. This paper focuses on the learning perspective of Self Organizing Networks (SON) solutions and provides, not only an overview of the most common ML techniques encountered in cellular networks, but also manages to classify each paper in terms of its learning solution, while also giving some examples. The authors also classify each paper in terms of its self-organizing use-case and discuss how each proposed solution performed. In addition, a comparison between the most commonly found ML algorithms in terms of certain SON metrics is performed and general guidelines on when to choose each ML algorithm for each SON function are proposed. Lastly, this work also provides future research directions and new paradigms that the use of more robust and intelligent algorithms, together with data gathered by operators, can bring to the cellular networks domain and fully enable the concept of SON in the near future
A Transfer Learning Approach for UAV Path Design with Connectivity Outage Constraint
The connectivity-aware path design is crucial in the effective deployment of
autonomous Unmanned Aerial Vehicles (UAVs). Recently, Reinforcement Learning
(RL) algorithms have become the popular approach to solving this type of
complex problem, but RL algorithms suffer slow convergence. In this paper, we
propose a Transfer Learning (TL) approach, where we use a teacher policy
previously trained in an old domain to boost the path learning of the agent in
the new domain. As the exploration processes and the training continue, the
agent refines the path design in the new domain based on the subsequent
interactions with the environment. We evaluate our approach considering an old
domain at sub-6 GHz and a new domain at millimeter Wave (mmWave). The teacher
path policy, previously trained at sub-6 GHz path, is the solution to a
connectivity-aware path problem that we formulate as a constrained Markov
Decision Process (CMDP). We employ a Lyapunov-based model-free Deep Q-Network
(DQN) to solve the path design at sub-6 GHz that guarantees connectivity
constraint satisfaction. We empirically demonstrate the effectiveness of our
approach for different urban environment scenarios. The results demonstrate
that our proposed approach is capable of reducing the training time
considerably at mmWave.Comment: 14 pages,8 figures, journal pape
Self-optimizing Strategies for Dynamic Vertical Sectorization in LTE Networks
International audience—Vertical Sectorization (VS) consists in creating ver-tically separated sectors in the original cell using an Active Antenna Systems (AAS) supporting two distinct beams with different downtilts. The total transmit power is split between the two sectors, while the frequency bandwidth can be reused by each sector, creating additional interference between the two sectors. For low traffic demand, VS may lead to performance degradation, while for high traffic demand in both sectors, VS is likely to bring about important capacity gains. Hence intelligent activation policy of VS is needed to fully benefit from this feature. In this paper, we propose an approach taking advantage of the more focused downtilted beam. A dynamic alpha fair bandwidth sharing is proposed for low and medium load. It is autonomously replaced by full bandwidth reuse for high load scenarios using a threshold-based controller. A flow-level dynamic simulator is used to numerically validate the proposed mechanisms
- …