75 research outputs found
5G Infrastructure Network Slicing: E2E Mean Delay Model and Effectiveness Assessment to Reduce Downtimes in Industry 4.0
This work has been partially funded by the H2020 project 5G-CLARITY (Grant No. 871428) and the Spanish national project TRUE-5G (PID2019-108713RB-C53).Fifth Generation (5G) is expected to meet stringent performance network requisites of
the Industry 4.0. Moreover, its built-in network slicing capabilities allow for the support of the
traffic heterogeneity in Industry 4.0 over the same physical network infrastructure. However, 5G
network slicing capabilities might not be enough in terms of degree of isolation for many private
5G networks use cases, such as multi-tenancy in Industry 4.0. In this vein, infrastructure network
slicing, which refers to the use of dedicated and well isolated resources for each network slice at every
network domain, fits the necessities of those use cases. In this article, we evaluate the effectiveness of
infrastructure slicing to provide isolation among production lines (PLs) in an industrial private 5G
network. To that end, we develop a queuing theory-based model to estimate the end-to-end (E2E)
mean packet delay of the infrastructure slices. Then, we use this model to compare the E2E mean
delay for two configurations, i.e., dedicated infrastructure slices with segregated resources for each
PL against the use of a single shared infrastructure slice to serve the performance-sensitive traffic
from PLs. Also we evaluate the use of Time-Sensitive Networking (TSN) against bare Ethernet to
provide layer 2 connectivity among the 5G system components. We use a complete and realistic
setup based on experimental and simulation data of the scenario considered. Our results support the
effectiveness of infrastructure slicing to provide isolation in performance among the different slices.
Then, using dedicated slices with segregated resources for each PL might reduce the number of the
production downtimes and associated costs as the malfunctioning of a PL will not affect the network
performance perceived by the performance-sensitive traffic from other PLs. Last, our results show
that, besides the improvement in performance, TSN technology truly provides full isolation in the
transport network compared to standard Ethernet thanks to traffic prioritization, traffic regulation,
and bandwidth reservation capabilities.H2020 project 5G-CLARITY 871428Spanish Government PID2019-108713RB-C53TRUE-5
On the Pricing of Forward Starting Options under Stochastic Volatility
We consider the problem of pricing European forward starting options in the presence of stochastic volatility. By performing a change of measure using the asset price at the time of strike determination as a numeraire, we derive a closed-form solution based on Heston’s model of stochastic volatility
Adaptive reinforcement learning for heterogeneous network selection
Next generation 5G mobile wireless networks will consist of multiple technologies for devices
to access the network at the edge. One of the keys to 5G is therefore the ability for
device to intelligently select its Radio Access Technology (RAT). Current fully distributed
algorithms for RAT selection although guaranteeing convergence to equilibrium states,
are often slow, require high exploration times and may converge to undesirable equilibria.
In this dissertation, we propose three novel reinforcement learning (RL) frameworks
to improve the efficiency of existing distributed RAT selection algorithms in a heterogeneous
environment, where users may potentially apply a number of different RAT selection
procedures. Although our research focuses on solutions for RAT selection in the
current and future mobile wireless networks, the proposed solutions in this dissertation
are general and suitable to apply for any large scale distributed multi-agent systems.
In the first framework, called RL with Non-positive Regret, we propose a novel adaptive
RL for multi-agent non-cooperative repeated games. The main contribution is to use both
positive and negative regrets in RL to improve the convergence speed and fairness of
the well-known regret-based RL procedure. Significant improvements in performance
compared to other related algorithms in the literature are demonstrated.
In the second framework, called RL with Network-Assisted Feedback (RLNF), our core
contribution is to develop a network feedback model that uses network-assisted information
to improve the performance of the distributed RL for RAT selection. RLNF guarantees
no-regret payoff in the long-run for any user adopting it, regardless of what other users
might do and so can work in an environment where not all users use the same learning
strategy. This is an important implementation advantage as RLNF can be implemented
within current mobile network standards.
In the third framework, we propose a novel adaptive RL-based mechanism for RAT selection
that can effectively handle user mobility. The key contribution is to leverage forgetting
methods to rapidly react to the changes in the radio conditions when users move.
We show that our solution improves the performance of wireless networks and converges
much faster when users move compared to the non-adaptive solutions. Another objective of the research is to study the impact of various network models on the
performance of different RAT selection approaches. We propose a unified benchmark to
compare the performances of different algorithms under the same computational environment.
The comparative studies reveal that among all the important network parameters
that influence the performance of RAT selection algorithms, the number of base stations
that a user can connect to has the most significant impact. This finding provides some
guidelines for the proper design of RAT selection algorithms for future 5G. Our evaluation
benchmark can serve as a reference for researchers, network developers, and engineers.
Overall, the thesis provides different reinforcement learning frameworks to improve the
efficiency of current fully distributed algorithms for heterogeneous RAT selection. We
prove the convergence of the proposed reinforcement learning procedures using the differential
inclusion (DI) technique. The theoretical analyses demonstrate that the use of
DI not only provides an effective method to study the convergence properties of adaptive
procedures in game-theoretic learning, but also yields a much more concise and extensible
proof as compared to the classical approaches.Thesis (Ph.D.) -- University of Adelaide, School of Electrical and Electronic Engineering, 201
Stochastic models for cognitive radio networks
During the last decade we have seen an explosive development of wireless technologies. Consequently the demand for electromagnetic spectrum has been growing dramatically resulting in the spectrum scarcity problem. In spite of this, spectrum utilization measurements have shown that licensed bands are vastly underutilized while unlicensed bands are too crowded. In this context, Cognitive Radio emerges as an auspicious paradigm in order to solve those problems. Even more, this concept is envisaged as one of the main components of future wireless technologies, such as the fifth-generation of mobile networks. In this regard, this thesis is founded on cognitive radio networks. We start considering a paid spectrum sharing approach where secondary users (SUs) pay to primary ones for the spectrum utilization. In particular, the first part of the thesis bears on the design and analysis of an optimal SU admission control policy, i.e. that maximizes the long-run profit of the primary service provider. We model the optimal revenue problem as a Markov Decision Process and we use dynamic programming (and other techniques such as sample-path analysis) to characterize properties of the optimal admission control policy. We introduce different changes to one of the best known dynamic programming algorithms incorporating the knowledge of the characterization. In particular, those proposals accelerate the rate of convergence of the algorithm when is applied in the considered context. We complement the analysis of the paid spectrum sharing approach using fluid approximations. That is to say, we obtain a description of the asymptotic behavior of the Markov process as the solution of an ordinary differential equation system. By means of the fluid approximation of the problem, we propose a methodology to estimate the optimal admission control boundary of the maximization profit problem mentioned before. In addition, we use the deterministic model in order to propose some tools and criteria that can be used to improve the mean spectrum utilization with the commitment of providing to secondary users certain quality of service levels. In wireless networks, a cognitive user can take advantage of either the time, the frequency, or the space. In the first part of the thesis we have been concentrated on timefrequency holes, in the second part we address the complete problem incorporating the space variable. In particular, we first introduce a probabilistic model based on a stochastic geometry approach. We focus our study in two of the main performance metrics: medium access probability and coverage probability. Finally, in the last part of the thesis we propose a novel methodology based on configuration models for random graphs. With our proposal, we show that it is possible to calculate an analytic approximation of the medium access probability (both for PUs and, most importantly, SUs) in an arbitrary large heterogeneous random network. This performance metric gives an idea of the possibilities offered by cognitive radio to improve the spectrum utilization. The introduced robust method, as well as all the results of the thesis, are evaluated by several simulations for different network topologies, including real scenarios of primary network deployments. Keywords: Markov decision process, fluid limit, stochastic geometry, random graphs,dynamic spectrum assignment, cognitive radi
Reinforcement Learning in Self Organizing Cellular Networks
Self-organization is a key feature as cellular networks densify and become more heterogeneous, through the additional small cells such as pico and femtocells. Self- organizing networks (SONs) can perform self-configuration, self-optimization, and self-healing. These operations can cover basic tasks such as the configuration of a newly installed base station, resource management, and fault management in the network. In other words, SONs attempt to minimize human intervention where they use measurements from the network to minimize the cost of installation, configuration, and maintenance of the network. In fact, SONs aim to bring two main factors in play: intelligence and autonomous adaptability. One of the main requirements for achieving such goals is to learn from sensory data and signal measurements in networks. Therefore, machine learning techniques can play a major role in processing underutilized sensory data to enhance the performance of SONs.
In the first part of this dissertation, we focus on reinforcement learning as a viable approach for learning from signal measurements. We develop a general framework in heterogeneous cellular networks agnostic to the learning approach. We design multiple reward functions and study different effects of the reward function, Markov state model, learning rate, and cooperation methods on the performance of reinforcement learning in cellular networks. Further, we look into the optimality of reinforcement learning solutions and provide insights into how to achieve optimal solutions.
In the second part of the dissertation, we propose a novel architecture based on spatial indexing for system-evaluation of heterogeneous 5G cellular networks. We develop an open-source platform based on the proposed architecture that can be used to study large scale directional cellular networks. The proposed platform is used for generating training data sets of accurate signal-to-interference-plus-noise-ratio (SINR) values in millimeter-wave communications for machine learning purposes. Then, with taking advantage of the developed platform, we look into dense millimeter-wave networks as one of the key technologies in 5G cellular networks. We focus on topology management of millimeter-wave backhaul networks and study and provide multiple insights on the evaluation and selection of proper performance metrics in dense millimeter-wave networks. Finally, we finish this part by proposing a self-organizing solution to achieve k-connectivity via reinforcement learning in the topology management of wireless networks
- …