Reinforcement Learning Based Handoff for Millimeter Wave Heterogeneous Cellular Networks

The millimeter wave (mmWave) radio band is promising for the next-generation heterogeneous cellular networks (HetNets) due to its large bandwidth available for meeting the increasing demand of mobile traffic. However, the unique propagation characteristics at mmWave band cause huge redundant handoffs in mmWave HetNets if conventional Reference Signal Received Power (RSRP) based handoff mechanism is used. In this paper, we propose a reinforcement learning based handoff policy named LESH to reduce the number of handoffs while maintaining user Quality of Service (QoS) requirements in mmWave HetNets. In LESH, we determine handoff trigger conditions by taking into account both mmWave channel characteristics and QoS requirements of UEs. Furthermore, we propose reinforcement-learning based BS selection algorithms for different UE densities. Numerical results show that in typical scenarios, LESH can significantly reduce the number of handoffs when compared with traditional handoff policies

The SMART handoff policy for millimeter wave heterogeneous cellular networks

The millimeter wave (mmWave) radio band is promising for the next-generation heterogeneous cellular networks (HetNets) due to its large bandwidth available for meeting the increasing demand of mobile traffic. However, the unique propagation characteristics at mmWave band cause huge redundant handoffs in mmWave HetNets that brings heavy signaling overhead, low energy efficiency and increased user equipment (UE) outage probability if conventional Reference Signal Received Power (RSRP) based handoff mechanism is used. In this paper, we propose a reinforcement learning based handoff policy named SMART to reduce the number of handoffs while maintaining user Quality of Service (QoS) requirements in mmWave HetNets. In SMART, we determine handoff trigger conditions by taking into account both mmWave channel characteristics and QoS requirements of UEs. Furthermore, we propose reinforcement-learning based BS selection algorithms for different UE densities. Numerical results show that in typical scenarios, SMART can significantly reduce the number of handoffs when compared with traditional handoff policies without learning

User Behavior Aware Cell Association in Heterogeneous Cellular Networks

In heterogeneous cellular networks (HetNets), cell association of User Equipment (UE) affects UE transmit rate and network throughput. Conventional cell association rules are usually based on UE received Signal-to-Interference-and-Noise-Ratio (SINR) without taking into account user behaviors, which can indeed be exploited for improving network performance. In this paper, we investigate UE cell association in HetNets based on individual user behavior characteristics with aim to maximize long- term expected system throughput. We model the problem as a stochastic optimization model Restless Multi-Armed Bandit (RMAB). As it is a PSPACE-hard problem, we develop a primal-dual heuristic index algorithm and the solution specifies the rule that determines which arms in the RMAB model to be selected at each decision time. According to the solution of RMAB, we propose a new cell association strategy called Index Enabled Association (IDEA). We also conduct simulation experiments to compare IDEA with conventional max-SINR cell association strategy and an existing game-based RAT selection scheme. Numerical results demonstrate the advantages of IDEA in typical scenarios

A Vision and Framework for the High Altitude Platform Station (HAPS) Networks of the Future

A High Altitude Platform Station (HAPS) is a network node that operates in the stratosphere at an of altitude around 20 km and is instrumental for providing communication services. Precipitated by technological innovations in the areas of autonomous avionics, array antennas, solar panel efficiency levels, and battery energy densities, and fueled by flourishing industry ecosystems, the HAPS has emerged as an indispensable component of next-generations of wireless networks. In this article, we provide a vision and framework for the HAPS networks of the future supported by a comprehensive and state-of-the-art literature review. We highlight the unrealized potential of HAPS systems and elaborate on their unique ability to serve metropolitan areas. The latest advancements and promising technologies in the HAPS energy and payload systems are discussed. The integration of the emerging Reconfigurable Smart Surface (RSS) technology in the communications payload of HAPS systems for providing a cost-effective deployment is proposed. A detailed overview of the radio resource management in HAPS systems is presented along with synergistic physical layer techniques, including Faster-Than-Nyquist (FTN) signaling. Numerous aspects of handoff management in HAPS systems are described. The notable contributions of Artificial Intelligence (AI) in HAPS, including machine learning in the design, topology management, handoff, and resource allocation aspects are emphasized. The extensive overview of the literature we provide is crucial for substantiating our vision that depicts the expected deployment opportunities and challenges in the next 10 years (next-generation networks), as well as in the subsequent 10 years (next-next-generation networks).Comment: To appear in IEEE Communications Surveys & Tutorial

Efficient handover mechanism for radio access network slicing by exploiting distributed learning

Network slicing is identified as a fundamental architectural technology for future mobile networks since it can logically separate networks into multiple slices and provide tailored quality of service (QoS). However, the introduction of network slicing into radio access networks (RAN) can greatly increase user handover complexity in cellular networks. Specifically, both physical resource constraints on base stations (BSs) and logical connection constraints on network slices (NSs) should be considered when making a handover decision. Moreover, various service types call for an intelligent handover scheme to guarantee the diversified QoS requirements. As such, in this paper, a multi-agent reinforcement LEarning based Smart handover Scheme, named LESS, is proposed, with the purpose of minimizing handover cost while maintaining user QoS. Due to the large action space introduced by multiple users and the data sparsity caused by user mobility, conventional reinforcement learning algorithms cannot be applied directly. To solve these difficulties, LESS exploits the unique characteristics of slicing in designing two algorithms: 1) LESS-DL, a distributed Q-learning algorithm to make handover decisions with reduced action space but without compromising handover performance; 2) LESS-QVU, a modified Q-value update algorithm which exploits slice traffic similarity to improve the accuracy of Q-value evaluation with limited data. Thus, LESS uses LESS-DL to choose the target BS and NS when a handover occurs, while Q-values are updated by using LESS-QVU. The convergence of LESS is theoretically proved in this paper. Simulation results show that LESS can significantly improve network performance. In more detail, the number of handovers, handover cost and outage probability are reduced by around 50%, 65%, and 45%, respectively, when compared with traditional methods