Decentralized Delay Optimal Control for Interference Networks with Limited Renewable Energy Storage

In this paper, we consider delay minimization for interference networks with renewable energy source, where the transmission power of a node comes from both the conventional utility power (AC power) and the renewable energy source. We assume the transmission power of each node is a function of the local channel state, local data queue state and local energy queue state only. In turn, we consider two delay optimization formulations, namely the decentralized partially observable Markov decision process (DEC-POMDP) and Non-cooperative partially observable stochastic game (POSG). In DEC-POMDP formulation, we derive a decentralized online learning algorithm to determine the control actions and Lagrangian multipliers (LMs) simultaneously, based on the policy gradient approach. Under some mild technical conditions, the proposed decentralized policy gradient algorithm converges almost surely to a local optimal solution. On the other hand, in the non-cooperative POSG formulation, the transmitter nodes are non-cooperative. We extend the decentralized policy gradient solution and establish the technical proof for almost-sure convergence of the learning algorithms. In both cases, the solutions are very robust to model variations. Finally, the delay performance of the proposed solutions are compared with conventional baseline schemes for interference networks and it is illustrated that substantial delay performance gain and energy savings can be achieved

Age Minimization in Energy Harvesting Communications: Energy-Controlled Delays

We consider an energy harvesting source that is collecting measurements from a physical phenomenon and sending updates to a destination within a communication session time. Updates incur transmission delays that are function of the energy used in their transmission. The more transmission energy used per update, the faster it reaches the destination. The goal is to transmit updates in a timely manner, namely, such that the total age of information is minimized by the end of the communication session, subject to energy causality constraints. We consider two variations of this problem. In the first setting, the source controls the number of measurement updates, their transmission times, and the amounts of energy used in their transmission (which govern their delays, or service times, incurred). In the second setting, measurement updates externally arrive over time, and therefore the number of updates becomes fixed, at the expense of adding data causality constraints to the problem. We characterize age-minimal policies in the two settings, and discuss the relationship of the age of information metric to other metrics used in the energy harvesting literature.Comment: Appeared in Asilomar 201