Contextual Bandit Approach for Energy Saving and Interference Coordination in HetNets

This paper addresses the joint problem of energy saving and interference coordination in heterogeneous networks (HetNets) using a contextual bandit formulation. We propose a semi-distributed scheme consisting of a learning agent and local controllers. The learning agent comprises a neural network (NN) classifier and a Multi-Armed Bandit (MAB) algorithm. The NN classifier is dynamically trained to choose a subset of configurations (i.e., feasible configurations in terms of QoS) based on the context information (network state). Then, the MAB algorithm picks one control (i.e., global configuration parameters) among those selected by the NN classifier, with the aim of improving the energy efficiency. These global configurations are interpreted by the local controllers on each network sector. This scheme allows the learning agent to progressively learn the best policy by observing the network state and the performance of the chosen configurations in terms of energy consumption and QoS. Our numerical results show an energy saving close to 20% with respect to a default policy and an improvement of 13% with respect to addressing energy saving and interference coordination separately

Online learning for self-optimization in heterogeneous networks

[SPA] Esta tesis doctoral se presenta bajo la modalidad de compendio de publicaciones. [ENG] This doctoral dissertation has been presented in the form of thesis by publication. These problems have been addressed by means of interference coordination (IC) and energy saving (ES) mechanisms. Although the configuration of these two mechanisms has been addressed separately so far, we show in this thesis that they are highly coupled. Moreover, the configuration of IC and ES is commonly addressed using network models, which presents several limitations. In this thesis, we consider the self-optimization functionality within the Self-Organizing Networks (SON) paradigm, which is intended to address these problems by allowing the network to autonomously configure its parameters while it is operating. To implement the self-optimization functionality, we propose the use of online learning algorithms, which learn efficient network configurations from experience without explicitly knowing the accurate mathematical model of the network beforehand. The first part of this thesis addresses the configuration of the IC mechanism in HetNets. We propose several online learning model-free solutions based on different techniques such as Response Surface Method (RSM) and Multi-Armed Bandit (MAB) algorithms. We also consider stochastic constraints in the learning process. In the second part, we address the joint problem of IC and ES in HetNets proposing several solutions based on Dynamic Programming, Contextual Multi-Armed Bandit algorithms, and Machine Learning tools such as Neural Networks and Gaussian processes.Los artículos que componen la tesis son los siguientes: 1. Jose A. Ayala-Romero, J. J. Alcaraz, J. Vales-Alonso, and E. Egea-López,“Online learning for interference coordination in heterogeneous networks”. IEEE International Conference on Communications (ICC). Paris (France), May 2017, pp. 1-6. DOI: 10.1109/ICC.2017.7996441. 2. Jose A. Ayala-Romero, J. J. Alcaraz, J. Vales-Alonso, and E. Egea-López, “Online Optimization of Interference Coordination Parameters in Small Cell Networks,” IEEE Transactions on Wireless Communications, vol. 16, no. 10, pp. 6635-6647, July 2017. DOI: 10.1109/TWC.2017.2727483. 3. Jose A. Ayala-Romero, J. J. Alcaraz, and J. Vales-Alonso, “Data-driven configuration of interference coordination parameters in HetNets,” IEEE Transactions on Vehicular Technology, vol. 67, no. 6, pp. 5174-5187, 2018. DOI: 10.1109/TVT.2018.2825606. 4. J. A. Ayala-Romero, J. J. Alcaraz, and J. Vales-Alonso, “Energy Saving and Interference Coordination in HetNets Using Dynamic Programming and CEC,” IEEE Access, vol. 6, pp. 71 110-71 121, 2018. DOI: 10.1109/ACCESS.2018.2881073. 5. Jose A. Ayala-Romero, J. J. Alcaraz, A. Zanella, and M. Zorzi, “Contextual bandit approach for energy saving and interference coordination in HetNets,” in IEEE International Conference on Communications (ICC) 2018. Kansas City (USA), May 2018, pp. 1-6. DOI: 10.1109/ICC.2018.8422872. 6. Jose A. Ayala-Romero, J. J. Alcaraz, A. Zanella, and M. Zorzi, “Online learning for energy saving and interference coordination in HetNets,” IEEE Journal on Selected Areas in Communications, vol. 37, no. 6, pp. 1374-1388, 2019. DOI: 10.1109/JSAC.2019.2904362.Escuela Internacional de Doctorado de la Universidad Politécnica de CartagenaUniversidad Politécnica de CartagenaPrograma de Doctorado en Tecnologías de la Información y las Comunicaciones por la Universidad Politécnica de Cartagen