Ensemble Reinforcement Learning: A Survey

Reinforcement Learning (RL) has emerged as a highly effective technique for addressing various scientific and applied problems. Despite its success, certain complex tasks remain challenging to be addressed solely with a single model and algorithm. In response, ensemble reinforcement learning (ERL), a promising approach that combines the benefits of both RL and ensemble learning (EL), has gained widespread popularity. ERL leverages multiple models or training algorithms to comprehensively explore the problem space and possesses strong generalization capabilities. In this study, we present a comprehensive survey on ERL to provide readers with an overview of recent advances and challenges in the field. First, we introduce the background and motivation for ERL. Second, we analyze in detail the strategies that have been successfully applied in ERL, including model averaging, model selection, and model combination. Subsequently, we summarize the datasets and analyze algorithms used in relevant studies. Finally, we outline several open questions and discuss future research directions of ERL. By providing a guide for future scientific research and engineering applications, this survey contributes to the advancement of ERL.Comment: 42 page

Machine Learning Research Trends in Africa: A 30 Years Overview with Bibliometric Analysis Review

In this paper, a critical bibliometric analysis study is conducted, coupled with an extensive literature survey on recent developments and associated applications in machine learning research with a perspective on Africa. The presented bibliometric analysis study consists of 2761 machine learning-related documents, of which 98% were articles with at least 482 citations published in 903 journals during the past 30 years. Furthermore, the collated documents were retrieved from the Science Citation Index EXPANDED, comprising research publications from 54 African countries between 1993 and 2021. The bibliometric study shows the visualization of the current landscape and future trends in machine learning research and its application to facilitate future collaborative research and knowledge exchange among authors from different research institutions scattered across the African continent

Multi-agent DRL for resource allocation and cache design in terrestrial-satellite networks

In the past few years, satellite communications have greatly affected our daily lives, and the integrated terrestrial-satellite network can combine the advantages of satellite and base stations (BSs) to provide wider coverage and lower cost. Because the resources of terrestrial-satellite network are limited, how to allocate resources of terrestrial-satellite network through effective methods has become a major challenge. This paper proposes a framework for resource allocation of terrestrial-satellite network based on non-orthogonal multiple access (NOMA). Then, a deployment method of local cache pools is given to achieve lower time delay and maximize energy efficiency in terrestrial-satellite network. In the proposed framework, we adopt a multi-agent deep deterministic policy gradient (MADDPG) method to obtain the maximum energy efficiency by user association, power control, and cache design. The MADDPG algorithm is divided into two stages, users and BSs are set as agents to complete the optimization problem in the framework. Finally, the simulation results show that the proposed method has better optimized performance compared with the traditional single-agent deep reinforcement learning algorithm and can efficiently solve the problems of resource allocation and cache design in the integrated terrestrial-satellite network

Neural network aided computation of mutual information for adaptation of spatial modulation

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Index Modulations, in the form of Spatial Modulation or Polarized Modulation, are gaining traction for both satellite and terrestrial next generation communication systems. Adaptive Spatial Modulation based links are needed to fully exploit the transmission capacity of time-variant channels. The adaptation of code and/or modulation requires a real-time evaluation of the channel achievable rates. Some existing results in the literature present a computational complexity which scales quadratically with the number of transmit antennas and the constellation order. Moreover, the accuracy of these approximations is low and it can lead to wrong Modulation and Coding Scheme selection. In this work we apply a Multilayer Feedforward Neural Network to compute the achievable rate of a generic Index Modulation link. The case of two antennas/polarizations is analyzed in depth, showing not only a one-hundred fold decrement of the Mean Square Error in the estimation of the capacity as compared with existing analytical approximations, but also a fifty times reduction of the computational complexity. Moreover, the extension to an arbitrary number of antennas is explained and supported with simulations. More generally, neural networks can be considered as promising candidates for the practical estimation of complex metrics in communication related settings.This work was funded by the Xunta de Galicia (Secretaria Xeral de Universidades) under a predoctoral scholarship (cofunded by the European Social Fund) and it was partially funded by the Agencia Estatal de Investigación (Spain) and the European Regional Development Fund (ERDF) under project MYRADA (TEC2016-75103-C2-2-R). It was also funded by the Xunta de Galicia and the ERDF (Agrupación Estratéxica Consolidada de Galicia accreditation 2016-2019). Furthermore, this work has received funding from the Spanish Agencia Estatal de Investigación under project TERESA, TEC2017-90093-C3-1-R (AEI/FEDER,UE); and from the Catalan Government (2017 SGR 891 and 2017 SGR 1479).Peer ReviewedPostprint (author's final draft

Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey

Non-terrestrial networks (NTNs) traditionally have certain limited applications. However, the recent technological advancements and manufacturing cost reduction opened up myriad applications of NTNs for 5G and beyond networks, especially when integrated into terrestrial networks (TNs). This article comprehensively surveys the evolution of NTNs highlighting their relevance to 5G networks and essentially, how it will play a pivotal role in the development of 6G ecosystem. We discuss important features of NTNs integration into TNs and the synergies by delving into the new range of services and use cases, various architectures, technological enablers, and higher layer aspects pertinent to NTNs integration. Moreover, we review the corresponding challenges arising from the technical peculiarities and the new approaches being adopted to develop efficient integrated ground-air-space (GAS) networks. Our survey further includes the major progress and outcomes from academic research as well as industrial efforts representing the main industrial trends, field trials, and prototyping towards the 6G networks

Evolution of Non-Terrestrial Networks From 5G to 6G: A Survey

Non-terrestrial networks (NTNs) traditionally have certain limited applications. However, the recent technological advancements and manufacturing cost reduction opened up myriad applications of NTNs for 5G and beyond networks, especially when integrated into terrestrial networks (TNs). This article comprehensively surveys the evolution of NTNs highlighting their relevance to 5G networks and essentially, how it will play a pivotal role in the development of 6G ecosystem. We discuss important features of NTNs integration into TNs and the synergies by delving into the new range of services and use cases, various architectures, technological enablers, and higher layer aspects pertinent to NTNs integration. Moreover, we review the corresponding challenges arising from the technical peculiarities and the new approaches being adopted to develop efficient integrated ground-air-space (GAS) networks. Our survey further includes the major progress and outcomes from academic research as well as industrial efforts representing the main industrial trends, field trials, and prototyping towards the 6G networks

A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

Due to the recent progress in Deep Neural Networks, Reinforcement Learning (RL) has become one of the most important and useful technology. It is a learning method where a software agent interacts with an unknown environment, selects actions, and progressively discovers the environment dynamics. RL has been effectively applied in many important areas of real life. This article intends to provide an in-depth introduction of the Markov Decision Process, RL and its algorithms. Moreover, we present a literature review of the application of RL to a variety of fields, including robotics and autonomous control, communication and networking, natural language processing, games and self-organized system, scheduling management and configuration of resources, and computer vision