Energy harvesting-aware design of wireless networks

Recent advances in low-power electronics and energy-harvesting (EH) technologies enable the design of self-sustained devices that collect part, or all, of the needed energy from the environment. Several systems can take advantage of EH, ranging from portable devices to wireless sensor networks (WSNs). While conventional design for battery-powered systems is mainly concerned with the battery lifetime, a key advantage of EH is that it enables potential perpetual operation of the devices, without requiring maintenance for battery substitutions. However, the inherent unpredictability regarding the amount of energy that can be collected from the environment might cause temporary energy shortages, which might prevent the devices to operate regularly. This uncertainty calls for the development of energy management techniques that are tailored to the EH dynamics. While most previous work on EH-capable systems has focused on energy management for single devices, the main contributions of this dissertation is the analysis and design of medium access control (MAC) protocols for WSNs operated by EH-capable devices. In particular, the dissertation first considers random access MAC protocols for single-hop EH networks, in which a fusion center collects data from a set of nodes distributed in its surrounding. MAC protocols commonly used in WSNs, such as time division multiple access (TDMA), framed-ALOHA (FA) and dynamic-FA (DFA) are investigated in the presence of EH-capable devices. A new ALOHA-based MAC protocol tailored to EH-networks, referred to as energy group-DFA (EG-DFA), is then proposed. In EG-DFA nodes with similar energy availability are grouped together and access the channel independently from other groups. It is shown that EG-DFA significantly outperforms the DFA protocol. Centralized scheduling-based MAC protocols for single-hop EH-networks with communication resource constraints are considered next. Two main scenarios are addressed, namely: i) nodes exclusively powered via EH; ii) nodes powered by a hybrid energy storage system, which is composed by a non-rechargeable battery and a capacitor charged via EH. For the former case the goal is the maximization of the network throughput, while in the latter the aim is maximizing the lifetime of the non-rechargeable batteries. For both scenarios optimal scheduling policies are derived by assuming different levels of information available at the fusion center about the energy availability at the nodes. When optimal policies are not derived explicitly, suboptimal policies are proposed and compared with performance upper bounds. Energy management policies for single devices have been investigated as well by focusing on radio frequency identification (RFID) systems, when the latter are operated by enhanced RFID tags with energy harvesting capabilities

Optimal Sensing and Transmission in Energy Harvesting Sensor Networks

Sensor networks equipped with energy harvesting (EH) devices have attracted great attentions recently. Compared with conventional sensor networks powered by batteries, the energy harvesting abilities of the sensor nodes make sustainable and environment-friendly sensor networks possible. However, the random, scarce and non-uniform energy supply features also necessitate a completely different approach to energy management. A typical EH wireless sensor node consists of an EH module that converts ambient energy to electrical energy, which is stored in a rechargeable battery, and will be used to power the sensing and transmission operations of the sensor. Therefore, both sensing and transmission are subject to the stochastic energy constraint imposed by the EH process. In this dissertation, we investigate optimal sensing and transmission policies for EH sensor networks under such constraints. For EH sensing, our objective is to understand how the temporal and spatial variabilities of the EH processes would affect the sensing performance of the network, and how sensor nodes should coordinate their data collection procedures with each other to cope with the random and non-uniform energy supply and provide reliable sensing performance with analytically provable guarantees. Specifically, we investigate optimal sensing policies for a single sensor node with infinite and finite battery sizes in Chapter 2, status updating/transmission strategy of an EH Source in Chapter 3, and a collaborative sensing policy for a multi-node EH sensor network in Chapter 4. For EH communication, our objective is to evaluate the impacts of stochastic variability of the EH process and practical battery usage constraint on the EH systems, and develop optimal transmission policies by taking such impacts into consideration. Specifically, we consider throughput optimization in an EH system under battery usage constraint in Chapter 5

Robust Restless Bandits: Tackling Interval Uncertainty with Deep Reinforcement Learning

We introduce Robust Restless Bandits, a challenging generalization of restless multi-arm bandits (RMAB). RMABs have been widely studied for intervention planning with limited resources. However, most works make the unrealistic assumption that the transition dynamics are known perfectly, restricting the applicability of existing methods to real-world scenarios. To make RMABs more useful in settings with uncertain dynamics: (i) We introduce the Robust RMAB problem and develop solutions for a minimax regret objective when transitions are given by interval uncertainties; (ii) We develop a double oracle algorithm for solving Robust RMABs and demonstrate its effectiveness on three experimental domains; (iii) To enable our double oracle approach, we introduce RMABPPO, a novel deep reinforcement learning algorithm for solving RMABs. RMABPPO hinges on learning an auxiliary "$\lambda$-network" that allows each arm's learning to decouple, greatly reducing sample complexity required for training; (iv) Under minimax regret, the adversary in the double oracle approach is notoriously difficult to implement due to non-stationarity. To address this, we formulate the adversary oracle as a multi-agent reinforcement learning problem and solve it with a multi-agent extension of RMABPPO, which may be of independent interest as the first known algorithm for this setting. Code is available at https://github.com/killian-34/RobustRMAB.Comment: 18 pages, 3 figure

Computation Offloading and Task Scheduling on Network Edge

The Fifth-Generation (5G) networks facilitate the evolution of communication systems and accelerate a revolution in the Information Technology (IT) field. In the 5G era, wireless networks are anticipated to provide connectivity for billions of Mobile User Devices (MUDs) around the world and to support a variety of innovative use cases, such as autonomous driving, ubiquitous Internet of Things (IoT), and Internet of Vehicles (IoV). The novel use cases, however, usually incorporate compute-intensive applications, which generate enormous computing service demands with diverse and stringent service requirements. In particular, autonomous driving calls for prompt data processing for the safety-related applications, IoT nodes deployed in remote areas need energy-efficient computing given limited on-board energy, and vehicles require low-latency computing for IoV applications in a highly dynamic network. To support the emerging computing service demands, Mobile Edge Computing (MEC), as a cutting-edge technology in 5G, utilizes computing resources on network edge to provide computing services for MUDs within a radio access network. The primary benefits of MEC can be elaborated from two perspectives. From the perspective of MUDs, MEC enables low-latency and energy-efficient computing by allowing MUDs to offload their computation tasks to proximal edge servers, which are installed in access points such as cellular base stations, Road-Side Units (RSUs), and Unmanned Aerial Vehicles (UAVs). On the other hand, from the perspective of network operators, MEC allows a large amount of computing data to be processed on network edge, thereby alleviating backhaul congestion. {MEC is a promising technology to support computing demands for the novel 5G applications within the RAN. The interesting issue is to maximize the computation capability of network edge to meet the diverse service requirements arising from the applications in dynamic network environments. However, the main technical challenges are: 1) how an edge server schedules its limited computing resources to optimize the Quality-of-Experience (QoE) in autonomous driving; 2) how the computation loads are balanced between the edge server and IoT nodes in computation loads to enable energy-efficient computing service provisioning; and 3) how multiple edge servers coordinate their computing resources to enable seamless and reliable computing services for high-mobility vehicles in IoV. In this thesis, we develop efficient computing resource management strategies for MEC, including computation offloading and task scheduling, to address the above three technical challenges. First, we study computation task scheduling to support real-time applications, such as localization and obstacle avoidance, for autonomous driving. In our considered scenario, autonomous vehicles periodically sense the environment, offload sensor data to an edge server for processing, and receive computing results from the edge server. Due to mobility and computing latency, a vehicle travels a certain distance between the instant of offloading its sensor data and the instant of receiving the computing result. Our objective is to design a scheduling scheme for the edge server to minimize the above traveled distance of vehicles. The idea is to determine the processing order according to the individual vehicle mobility and computation capability of the edge server. We formulate a Restless Multi-Armed Bandit (RMAB) problem, design a Whittle index-based stochastic scheduling scheme, and determine the index using a Deep Reinforcement Learning (DRL) method. The proposed scheduling scheme can avoid the time-consuming policy exploration common in DRL scheduling approaches and makes effectual decisions with low complexity. Extensive simulation results demonstrate that, with the proposed index-based scheme, the edge server can deliver computing results to the vehicles promptly while adapting to time-variant vehicle mobility. Second, we study energy-efficient computation offloading and task scheduling for an edge server while provisioning computing services {for IoT nodes in remote areas}. In the considered scenario, a UAV is equipped with computing resources and plays the role of an aerial edge server to collect and process the computation tasks offloaded by ground MUDs. Given the service requirements of MUDs, we aim to maximize UAV energy efficiency by jointly optimizing the UAV trajectory, the user transmit power, and computation task scheduling. The resulting optimization problem corresponds to nonconvex fractional programming, and the Dinkelbach algorithm and the Successive Convex Approximation (SCA) technique are adopted to solve it. Furthermore, we decompose the problem into multiple subproblems for distributed and parallel problem solving. To cope with the case when the knowledge of user mobility is limited, we apply a spatial distribution estimation technique to predict the location of ground users so that the proposed approach can still be valid. Simulation results demonstrate the effectiveness of the proposed approach to maximize the energy efficiency of the UAV. Third, we study collaboration among multiple edge servers in computation offloading and task scheduling to support computing services {in IoV}. In the considered scenario, vehicles traverse the coverage of edge servers and offload their tasks to their proximal edge servers. We develop a collaborative edge computing framework to reduce computing service latency and alleviate computing service interruption due to the high mobility of vehicles: 1) a Task Partition and Scheduling Algorithm (TPSA) is proposed to schedule the execution order of the tasks offloaded to the edge servers given a computation offloading strategy; and 2) an artificial intelligence-based collaborative computing approach is developed to determine the task offloading, computing, and result delivery policy for vehicles. Specifically, the offloading and computing problem is formulated as a Markov decision process. A DRL technique, i.e., deep deterministic policy gradient, is adopted to find the optimal solution in a complex urban transportation network. With the developed framework, the service cost, which includes computing service latency and service failure penalty, can be minimized via the optimal computation task scheduling and edge server selection. Simulation results show that the proposed AI-based collaborative computing approach can adapt to a highly dynamic environment with outstanding performance. In summary, we investigate computing resource management to optimize QoE of MUDs in the coverage of an edge server, to improve energy efficiency for an aerial edge server while provisioning computing services, and to coordinate computing resources among edge servers for supporting MUDs with high mobility. The proposed approaches and theoretical results contribute to computing resource management for MEC in 5G and beyond

Energy Harvesting Wireless Communications: A Review of Recent Advances

This article summarizes recent contributions in the broad area of energy harvesting wireless communications. In particular, we provide the current state of the art for wireless networks composed of energy harvesting nodes, starting from the information-theoretic performance limits to transmission scheduling policies and resource allocation, medium access and networking issues. The emerging related area of energy transfer for self-sustaining energy harvesting wireless networks is considered in detail covering both energy cooperation aspects and simultaneous energy and information transfer. Various potential models with energy harvesting nodes at different network scales are reviewed as well as models for energy consumption at the nodes.Comment: To appear in the IEEE Journal of Selected Areas in Communications (Special Issue: Wireless Communications Powered by Energy Harvesting and Wireless Energy Transfer