Energy Management in a Cooperative Energy Harvesting Wireless Sensor Network

In this paper, we consider the problem of finding an optimal energy management policy for a network of sensor nodes capable of harvesting their own energy and sharing it with other nodes in the network. We formulate this problem in the discounted cost Markov decision process framework and obtain good energy-sharing policies using the Deep Deterministic Policy Gradient (DDPG) algorithm. Earlier works have attempted to obtain the optimal energy allocation policy for a single sensor and for multiple sensors arranged on a mote with a single centralized energy buffer. Our algorithms, on the other hand, provide optimal policies for a distributed network of sensors individually harvesting energy and capable of sharing energy amongst themselves. Through simulations, we illustrate that the policies obtained by our DDPG algorithm using this enhanced network model outperform algorithms that do not share energy or use a centralized energy buffer in the distributed multi-nodal case.Comment: 11 pages, 4 figure

A cell outage management framework for dense heterogeneous networks

In this paper, we present a novel cell outage management (COM) framework for heterogeneous networks with split control and data planes-a candidate architecture for meeting future capacity, quality-of-service, and energy efficiency demands. In such an architecture, the control and data functionalities are not necessarily handled by the same node. The control base stations (BSs) manage the transmission of control information and user equipment (UE) mobility, whereas the data BSs handle UE data. An implication of this split architecture is that an outage to a BS in one plane has to be compensated by other BSs in the same plane. Our COM framework addresses this challenge by incorporating two distinct cell outage detection (COD) algorithms to cope with the idiosyncrasies of both data and control planes. The COD algorithm for control cells leverages the relatively larger number of UEs in the control cell to gather large-scale minimization-of-drive-test report data and detects an outage by applying machine learning and anomaly detection techniques. To improve outage detection accuracy, we also investigate and compare the performance of two anomaly-detecting algorithms, i.e., k-nearest-neighbor- and local-outlier-factor-based anomaly detectors, within the control COD. On the other hand, for data cell COD, we propose a heuristic Grey-prediction-based approach, which can work with the small number of UE in the data cell, by exploiting the fact that the control BS manages UE-data BS connectivity and by receiving a periodic update of the received signal reference power statistic between the UEs and data BSs in its coverage. The detection accuracy of the heuristic data COD algorithm is further improved by exploiting the Fourier series of the residual error that is inherent to a Grey prediction model. Our COM framework integrates these two COD algorithms with a cell outage compensation (COC) algorithm that can be applied to both planes. Our COC solution utilizes an actor-critic-based reinforcement learning algorithm, which optimizes the capacity and coverage of the identified outage zone in a plane, by adjusting the antenna gain and transmission power of the surrounding BSs in that plane. The simulation results show that the proposed framework can detect both data and control cell outage and compensate for the detected outage in a reliable manner

Robotic ubiquitous cognitive ecology for smart homes

Robotic ecologies are networks of heterogeneous robotic devices pervasively embedded in everyday environments, where they cooperate to perform complex tasks. While their potential makes them increasingly popular, one fundamental problem is how to make them both autonomous and adaptive, so as to reduce the amount of preparation, pre-programming and human supervision that they require in real world applications. The project RUBICON develops learning solutions which yield cheaper, adaptive and efficient coordination of robotic ecologies. The approach we pursue builds upon a unique combination of methods from cognitive robotics, machine learning, planning and agent- based control, and wireless sensor networks. This paper illustrates the innovations advanced by RUBICON in each of these fronts before describing how the resulting techniques have been integrated and applied to a smart home scenario. The resulting system is able to provide useful services and pro-actively assist the users in their activities. RUBICON learns through an incremental and progressive approach driven by the feed- back received from its own activities and from the user, while also self-organizing the manner in which it uses available sensors, actuators and other functional components in the process. This paper summarises some of the lessons learned by adopting such an approach and outlines promising directions for future work

220102

In wireless powered sensor networks (WPSN), data of ground sensors can be collected or relayed by an unmanned aerial vehicle (UAV) while the battery of the ground sensor can be charged via wireless power transfer. A key challenge of resource allocation in UAV-aided WPSN is to prevent battery drainage and buffer overflow of the ground sensors in the presence of highly dynamic lossy airborne channels which can result in packet reception errors. Moreover, state and action spaces of the resource allocation problem are large, which is hardly explored online. To address the challenges, a new data-driven deep reinforcement learning framework, DDRL-RA, is proposed to train flight resource allocation online so that the data packet loss is minimized. Due to time-varying airborne channels, DDRL-RA firstly leverages long short-term memory (LSTM) with pre-collected offline datasets for channel randomness predictions. Then, Deep Deterministic Policy Gradient (DDPG) is studied to control the flight trajectory of the UAV, and schedule the ground sensor to transmit data and harvest energy. To evaluate the performance of DDRL-RA, a UAV-ground sensor testbed is built, where real-world datasets of channel gains are collected. DDRL-RA is implemented on Tensorflow, and numerical results show that DDRL-RA achieves 19\% lower packet loss than other learning-based frameworks.This work was partially supported by National Funds through FCT/MCTES (Portuguese Foundation for Science and Technology), within the CISTER Research Unit (UIDP/UIDB/04234/2020); also by national funds through the FCT, under CMU Portugal partnership, within project CMU/TIC/0022/2019 (CRUAV). This work was in part supported by the Federal Ministry of Education and Research (BMBF, Germany) as part of the 6G Research and Innovation Cluster 6G-RIC under Grant 16KISK020K.info:eu-repo/semantics/publishedVersio