Cooperative relay selection for load balancing with mobility in hierarchical WSNs: A multi-armed bandit approach

© 2013 IEEE. Energy efficiency is the major concern in hierarchical wireless sensor networks(WSNs), where the major energy consumption originates from radios for communication. Due to notable energy expenditure of long-range transmission for cluster members and data aggregation for Cluster Head (CH), saving and balancing energy consumption is a tricky challenge in WSNs. In this paper, we design a CH selection mechanism with a mobile sink (MS) while proposing relay selection algorithms with multi-user multi-armed bandit (UM-MAB) to solve the problem of energy efficiency. According to the definition of node density and residual energy, we propose a conception referred to as a Virtual Head (VH) for MS to collect data in terms of energy efficiency. Moreover, we naturally change the relay selection problem into permutation problem through employing the two-hop transmission in cooperative power line communication, which deals with long-distance transmission. As far as the relay selection problem is concerned, we propose the machine learning algorithm, namely MU-MAB, to solve it through the reward associated with an increment for energy consumption. Furthermore, we employ the stable matching theory based on marginal utility for the allocation of the final one-to-one optimal combinations to achieve energy efficiency. In order to evaluate MU-MAB, the regret is taken advantage to demonstrate the performance by using upper confidence bound (UCB) index. In the end, simulation results illustrate the efficacy and effectiveness of our proposed solutions for saving and balancing energy consumption

Self-Evolving Integrated Vertical Heterogeneous Networks

6G and beyond networks tend towards fully intelligent and adaptive design in order to provide better operational agility in maintaining universal wireless access and supporting a wide range of services and use cases while dealing with network complexity efficiently. Such enhanced network agility will require developing a self-evolving capability in designing both the network architecture and resource management to intelligently utilize resources, reduce operational costs, and achieve the coveted quality of service (QoS). To enable this capability, the necessity of considering an integrated vertical heterogeneous network (VHetNet) architecture appears to be inevitable due to its high inherent agility. Moreover, employing an intelligent framework is another crucial requirement for self-evolving networks to deal with real-time network optimization problems. Hence, in this work, to provide a better insight on network architecture design in support of self-evolving networks, we highlight the merits of integrated VHetNet architecture while proposing an intelligent framework for self-evolving integrated vertical heterogeneous networks (SEI-VHetNets). The impact of the challenges associated with SEI-VHetNet architecture, on network management is also studied considering a generalized network model. Furthermore, the current literature on network management of integrated VHetNets along with the recent advancements in artificial intelligence (AI)/machine learning (ML) solutions are discussed. Accordingly, the core challenges of integrating AI/ML in SEI-VHetNets are identified. Finally, the potential future research directions for advancing the autonomous and self-evolving capabilities of SEI-VHetNets are discussed.Comment: 25 pages, 5 figures, 2 table

Pervasive AI for IoT applications: A Survey on Resource-efficient Distributed Artificial Intelligence

Artificial intelligence (AI) has witnessed a substantial breakthrough in a variety of Internet of Things (IoT) applications and services, spanning from recommendation systems and speech processing applications to robotics control and military surveillance. This is driven by the easier access to sensory data and the enormous scale of pervasive/ubiquitous devices that generate zettabytes of real-time data streams. Designing accurate models using such data streams, to revolutionize the decision-taking process, inaugurates pervasive computing as a worthy paradigm for a better quality-of-life (e.g., smart homes and self-driving cars.). The confluence of pervasive computing and artificial intelligence, namely Pervasive AI, expanded the role of ubiquitous IoT systems from mainly data collection to executing distributed computations with a promising alternative to centralized learning, presenting various challenges, including privacy and latency requirements. In this context, an intelligent resource scheduling should be envisaged among IoT devices (e.g., smartphones, smart vehicles) and infrastructure (e.g., edge nodes and base stations) to avoid communication and computation overheads and ensure maximum performance. In this paper, we conduct a comprehensive survey of the recent techniques and strategies developed to overcome these resource challenges in pervasive AI systems. Specifically, we first present an overview of the pervasive computing, its architecture, and its intersection with artificial intelligence. We then review the background, applications and performance metrics of AI, particularly Deep Learning (DL) and reinforcement learning, running in a ubiquitous system. Next, we provide a deep literature review of communication-efficient techniques, from both algorithmic and system perspectives, of distributed training and inference across the combination of IoT devices, edge devices and cloud servers. Finally, we discuss our future vision and research challenges

Unmanned Aerial Vehicle (UAV)-Enabled Wireless Communications and Networking

The emerging massive density of human-held and machine-type nodes implies larger traffic deviatiolns in the future than we are facing today. In the future, the network will be characterized by a high degree of flexibility, allowing it to adapt smoothly, autonomously, and efficiently to the quickly changing traffic demands both in time and space. This flexibility cannot be achieved when the network’s infrastructure remains static. To this end, the topic of UAVs (unmanned aerial vehicles) have enabled wireless communications, and networking has received increased attention. As mentioned above, the network must serve a massive density of nodes that can be either human-held (user devices) or machine-type nodes (sensors). If we wish to properly serve these nodes and optimize their data, a proper wireless connection is fundamental. This can be achieved by using UAV-enabled communication and networks. This Special Issue addresses the many existing issues that still exist to allow UAV-enabled wireless communications and networking to be properly rolled out

A Gentle Introduction to Reinforcement Learning and its Application in Different Fields

Due to the recent progress in Deep Neural Networks, Reinforcement Learning (RL) has become one of the most important and useful technology. It is a learning method where a software agent interacts with an unknown environment, selects actions, and progressively discovers the environment dynamics. RL has been effectively applied in many important areas of real life. This article intends to provide an in-depth introduction of the Markov Decision Process, RL and its algorithms. Moreover, we present a literature review of the application of RL to a variety of fields, including robotics and autonomous control, communication and networking, natural language processing, games and self-organized system, scheduling management and configuration of resources, and computer vision

Resource-optimized multiarmed bandit-based offload path selection in edge UAV swarms

This paper looks into the problem of a decentralized data offloading within an edge unmanned aerial vehicle (UAV) swarm to mitigate the complexities of a single UAV continually generating and processing large application-specific data. The mobile edge UAVs considered here are multirotor types having constrained energy and processing power, which makes long-term handling of large data volumes impossible for standalone UAVs. The load mitigation is carried out by offloading data from a source UAV to other swarm members with sufficient energy and processing requirements. In this paper, we focus on selecting the most optimal multihop path through the UAVs concerning available energies and processing resources, which can survive the duration of the data offload between the source and a target UAV. We formulate a multiarmed bandit-based offload path selection scheme, which selects the most energy and processing optimized multihop path between a source and a target UAV. Upon comparison of our scheme against the naive shortest path approach, we observe that our approach results in significant savings of collective network energies, even for long operational durations