A Reliable Reinforcement Learning for Resource Allocation in Uplink NOMA-URLLC Networks

In this paper, we propose a deep state-action-reward-state-action (SARSA) λ learning approach for optimising the uplink resource allocation in non-orthogonal multiple access (NOMA) aided ultra-reliable low-latency communication (URLLC). To reduce the mean decoding error probability in time-varying network environments, this work designs a reliable learning algorithm for providing a long-term resource allocation, where the reward feedback is based on the instantaneous network performance. With the aid of the proposed algorithm, this paper addresses three main challenges of the reliable resource sharing in NOMA-URLLC networks: 1) user clustering; 2) Instantaneous feedback system; and 3) Optimal resource allocation. All of these designs interact with the considered communication environment. Lastly, we compare the performance of the proposed algorithm with conventional Q-learning and SARSA Q-learning algorithms. The simulation outcomes show that: 1) Compared with the traditional Q learning algorithms, the proposed solution is able to converges within 200 episodes for providing as low as 10-2 long-term mean error; 2) NOMA assisted URLLC outperforms traditional OMA systems in terms of decoding error probabilities; and 3) The proposed feedback system is efficient for the long-term learning process

Modeling and Analysis of D2D Millimeter-Wave Networks With Poisson Cluster Processes

This paper investigates the performance of millimeter wave (mmWave) communications in clustered device-to-device (D2D) networks. The locations of D2D transceivers are modeled as a Poisson Cluster Process (PCP). In each cluster, devices are equipped with multiple antennas, and the active D2D transmitter (D2D-Tx) utilizes mmWave to serve one of the proximate D2D receivers (D2D-Rxs). Specifically, we introduce three user association strategies: 1) Uniformly distributed D2D-Tx model; 2) Nearest D2D-Tx model; 3) Closest line-of-site (LOS) D2D-Tx model. To characterize the performance of the considered scenarios, we derive new analytical expressions for the coverage probability and area spectral efficiency (ASE). Additionally, in order to efficiently illustrating the general trends of our system, a closed-form lower bound for the special case interfered by intra-cluster LOS links is derived. We provide Monte Carlo simulations to corroborate the theoretical results and show that: 1) The coverage probability is mainly affected by the intra-cluster interference with LOS links; 2) There exists an optimum number of simultaneously active D2D-Txs in each cluster for maximizing ASE; and 3) Closest LOS model outperforms the other two scenarios but at the cost of extra system overhead.Comment: This paper has been published in IEEE Transactions on Communications. Please cite the formal version of this pape

Modeling pulsar time noise with long term power law decay modulated by short term oscillations of the magnetic fields of neutron stars

We model the evolution of the magnetic fields of neutron stars as consisting of a long term power-law decay modulated by short term small amplitude oscillations. Our model predictions on the timing noise $\ddot\nu$ of neutron stars agree well with the observed statistical properties and correlations of normal radio pulsars. Fitting the model predictions to the observed data, we found that their initial parameter implies their initial surface magnetic dipole magnetic field strength ~ 5E14 G at ~0.4 year old and that the oscillations have amplitude between E-8 to E-5 and period on the order of years. For individual pulsars our model can effectively reduce their timing residuals, thus offering the potential of more sensitive detections of gravitational waves with pulsar timing arrays. Finally our model can also re-produce their observed correlation and oscillations of the second derivative of spin frequency, as well as the "slow glitch" phenomenon.Comment: 10 pages, 6 figures, submitted to IJMPD, invited talk in the 3rd Galileo-XuGuangqi Meeting}, Beijing, China, 12-16 October 201

Resource Allocation in Uplink NOMA-IoT Networks: A Reinforcement-Learning Approach

Non-orthogonal multiple access (NOMA) exploits the potential of the power domain to enhance the connectivity for the Internet of Things (IoT). Due to time-varying communication channels, dynamic user clustering is a promising method to increase the throughput of NOMA-IoT networks. This paper develops an intelligent resource allocation scheme for uplink NOMA-IoT communications. To maximise the average performance of sum rates, this work designs an efficient optimization approach based on two reinforcement learning algorithms, namely deep reinforcement learning (DRL) and SARSA-learning. For light traffic, SARSA-learning is used to explore the safest resource allocation policy with low cost. For heavy traffic, DRL is used to handle traffic-introduced huge variables. With the aid of the considered approach, this work addresses two main problems of fair resource allocation in NOMA techniques: 1) allocating users dynamically and 2) balancing resource blocks and network traffic. We analytically demonstrate that the rate of convergence is inversely proportional to network sizes. Numerical results show that: 1) Compared with the optimal benchmark scheme, the proposed DRL and SARSA-learning algorithms have lower complexity with acceptable accuracy and 2) NOMA-enabled IoT networks outperform the conventional orthogonal multiple access based IoT networks in terms of system throughput