User-centric association in ultra-dense mmWave networks via deep reinforcement learning

For ultra-dense networks, user-centric architecture is regarded as a promising candidate to offer mobile users better quality of service. One of the main challenges of user-centric architecture is exploring efficient scheme for user association in the ultra-dense network. In this letter, we study dynamic user-centric association (UCA) problem for ultra-dense millimeter wave (mmWave) networks to provide reliable connectivity and high achievable data rate. We consider time-varying network environments and propose a deep Q-network based UCA scheme to find the optimal association policy based on the historical experience. Simulation results are presented to verify the performance gain of our proposed scheme

Twin Delayed DDPG based Dynamic Power Allocation for Mobility in IoRT

The internet of robotic things (IoRT) is a modern as well as fast-evolving technology employed in abundant socio-economical aspects which connect user equipment (UE) for communication and data transfer among each other. For ensuring the quality of service (QoS) in IoRT applications, radio resources, for example, transmitting power allocation (PA), interference management, throughput maximization etc., should be efficiently employed and allocated among UE. Traditionally, resource allocation has been formulated using optimization problems, which are then solved using mathematical computer techniques. However, those optimization problems are generally nonconvex as well as nondeterministic polynomial-time hardness (NP-hard). In this paper, one of the most crucial challenges in radio resource management is the emitting power of an antenna called PA, considering that the interfering multiple access channel (IMAC) has been considered. In addition, UE has a natural movement behavior that directly impacts the channel condition between remote radio head (RRH) and UE. Additionally, we have considered two well-known UE mobility models i) random walk and ii) modified Gauss-Markov (GM). As a result, the simulation environment is more realistic and complex. A data-driven as well as model-free continuous action based deep reinforcement learning algorithm called twin delayed deep deterministic policy gradient (TD3) has been proposed that is the combination of policy gradient, actor-critics, as well as double deep Q-learning (DDQL). It optimizes the PA for i) stationary UE, ii) the UE movements according to random walk model, and ii) the UE movement based on the modified GM model. Simulation results show that the proposed TD3 method outperforms model-based techniques like weighted MMSE (WMMSE) and fractional programming (FP) as well as model-free algorithms, for example, deep Q network (DQN) and DDPG in terms of average sum-rate performance

Optimal learning paradigm and clustering for effective radio resource management in 5G HetNets

Ultra-dense heterogeneous networks (UDHN) based on small cells are a requisite part of the future cellular networks as they are proposed as one of the enabling technologies to handle coverage and capacity problems. But co-tier and cross-tier interferences in UDHN severely degrade the quality of service due to K-tiered architecture. Machine learning based radio resource management either through independent learning or cooperative learning is a proven efficient scheme for interference mitigation and quality of service provision in UDHN in a both distributive and cooperative manner. However, an optimal learning paradigm selection, i.e., either independent or cooperative learning and optimal cooperative cluster size in cooperative learning for efficient radio resource management in UDHN is still an open research problem. In this article, a Q-learning based radio resource management scheme is proposed and evaluated for both distributive and cooperative schemes using independent and cooperative learning. The proposed Q-learning solution follows the $\epsilon -$ greedy policy for optimal convergence. The simulation results for the UDHN in an urban setup show that in comparison to the independent learning paradigm, cooperative learning has no significant impact on macro cell user capacity. However, there is a significant improvement in small cell user capacity and the sum capacity of the cooperating small cells in the cluster. A significant increase of 48.57% and 37.9% is observed in the small cell user capacity, and sum capacity of the cooperating small cells, respectively, using cooperative learning as compared to independent learning which sets cooperative learning as an optimal learning strategy in UDHN. The improvement in small cell user capacity is at cost of increased computational time which is directly proportional to the number of cooperating small cells. To solve the issue of computational time in cooperative learning, an optimal clustering algorithm is proposed. The proposed optimal clustering reduced the computational time by four times in cooperative Q-learning