Centralized & Distributed Deep Reinforcement Learning Methods for Downlink Sum-Rate Optimization

For a multi-cell, multi-user, cellular network downlink sum-rate maximization through power allocation is a nonconvex and NP-hard optimization problem. In this paper, we present an effective approach to solving this problem through single- and multi-agent actor-critic deep reinforcement learning (DRL). Specifically, we use finite-horizon trust region optimization. Through extensive simulations, we show that we can simultaneously achieve higher spectral efficiency than state-of-the-art optimization algorithms like weighted minimum mean-squared error (WMMSE) and fractional programming (FP), while offering execution times more than two orders of magnitude faster than these approaches. Additionally, the proposed trust region methods demonstrate superior performance and convergence properties than the Advantage Actor-Critic (A2C) DRL algorithm. In contrast to prior approaches, the proposed decentralized DRL approaches allow for distributed optimization with limited CSI and controllable information exchange between BSs while offering competitive performance and reduced training times.Comment: Accepted for publication in IEEE Transactions on Wireless Communication

Optimizing Downlink Resource Allocation in Multiuser MIMO Networks via Fractional Programming and the Hungarian Algorithm

Optimizing the sum-log-utility for the downlink of multi-frequency band, multiuser, multiantenna networks requires joint solutions to the associated beamforming and user scheduling problems through the use of cloud radio access network (CRAN) architecture; optimizing such a network is, however, non-convex and NP-hard. In this paper, we present a novel iterative beamforming and scheduling strategy based on fractional programming and the Hungarian algorithm. The beamforming strategy allows us to iteratively maximize the chosen objective function in a fashion similar to block coordinate ascent. Furthermore, based on the crucial insight that, in the downlink, the interference pattern remains fixed for a given set of beamforming weights, we use the Hungarian algorithm as an efficient approach to optimally schedule users for the given set of beamforming weights. Specifically, this approach allows us to select the best subset of users (amongst the larger set of all available users). Our simulation results show that, in terms of average sum-log-utility, as well as sum-rate, the proposed scheme substantially outperforms both the state-of-the-art multicell weighted minimum mean-squared error (WMMSE) and greedy proportionally fair WMMSE schemes, as well as standard interior-point and sequential quadratic solvers. Importantly, our proposed scheme is also far more computationally efficient than the multicell WMMSE scheme.Comment: Accepted for publication in IEEE Transactions on Wireless Communications (2020