Finite Horizon Online Lazy Scheduling with Energy Harvesting Transmitters over Fading Channels

Lazy scheduling, i.e. setting transmit power and rate in response to data traffic as low as possible so as to satisfy delay constraints, is a known method for energy efficient transmission.This paper addresses an online lazy scheduling problem over finite time-slotted transmission window and introduces low-complexity heuristics which attain near-optimal performance.Particularly, this paper generalizes lazy scheduling problem for energy harvesting systems to deal with packet arrival, energy harvesting and time-varying channel processes simultaneously. The time-slotted formulation of the problem and depiction of its offline optimal solution provide explicit expressions allowing to derive good online policies and algorithms

Stochastic modelling of energy harvesting for low power sensor nodes

Battery lifetime is a key impediment to long-lasting low power sensor nodes. Energy or power harvesting mitigates the ependency on battery power, by converting ambient energy into electrical energy. This energy can then be used by the device for data collection and transmission. This paper proposes and analyses a queueing model to assess performance of such an energy harvesting sensor node. Accounting for energy harvesting, data collection and data transmission opportunities, the sensor node is modelled as a paired queueing system. The system has two queues, one representing accumulated energy and the other being the data queue. By means of some numerical examples, we investigate the energy-information trade-off

Energy Sharing for Multiple Sensor Nodes with Finite Buffers

We consider the problem of finding optimal energy sharing policies that maximize the network performance of a system comprising of multiple sensor nodes and a single energy harvesting (EH) source. Sensor nodes periodically sense the random field and generate data, which is stored in the corresponding data queues. The EH source harnesses energy from ambient energy sources and the generated energy is stored in an energy buffer. Sensor nodes receive energy for data transmission from the EH source. The EH source has to efficiently share the stored energy among the nodes in order to minimize the long-run average delay in data transmission. We formulate the problem of energy sharing between the nodes in the framework of average cost infinite-horizon Markov decision processes (MDPs). We develop efficient energy sharing algorithms, namely Q-learning algorithm with exploration mechanisms based on the $\epsilon$-greedy method as well as upper confidence bound (UCB). We extend these algorithms by incorporating state and action space aggregation to tackle state-action space explosion in the MDP. We also develop a cross entropy based method that incorporates policy parameterization in order to find near optimal energy sharing policies. Through simulations, we show that our algorithms yield energy sharing policies that outperform the heuristic greedy method.Comment: 38 pages, 10 figure

Wireless Throughput and Energy Efficiency under QoS Constraints

Mobile data traffic has experienced unprecedented growth recently and is predicted to grow even further over the coming years. As one of the main driving forces behind this growth, wireless transmission of multimedia content has significantly increased in volume and is expected to be the dominant traffic in data communications. Such wireless multimedia traffic requires certain quality-of-service (QoS) guarantees. With these motivations, in the first part of the thesis, throughput and energy efficiency in fading channels are studied in the presence of randomly arriving data and statistical queueing constraints. In particular, Markovian arrival models including discrete-time Markov, Markov fluid, and Markov-modulated Poisson sources are considered, and maximum average arrival rates in the presence of statistical queueing constraints are characterized. Furthermore, energy efficiency is analyzed by determining the minimum energy per bit and wideband slope in the low signal-to-noise ratio (SNR) regime. Following this analysis, energy-efficient power adaptation policies in fading channels are studied when data arrivals are modeled as Markovian processes and statistical QoS constraints are imposed. After formulating energy efficiency (EE) as maximum throughput normalized by the total power consumption, optimal power control policies that maximize EE are obtained for different source models. Next, throughput and energy efficiency of secure wireless transmission of delay sensitive data generated by random sources are investigated. A fading broadcast model in which the transmitter sends confidential and common messages to two receivers is considered. It is assumed that the common and confidential data, generated from Markovian sources, is stored in buffers prior to transmission, and the transmitter operates under constraints on buffer/delay violation probability. Under such statistical QoS constraints, the throughput is determined. In particular, secrecy capacity is used to describe the service rate of buffers containing confidential messages. Moreover, energy efficiency is studied in the low signal-to-noise (SNR) regime. In the final part of the thesis, throughput and energy efficiency are addressed considering the multiuser channel models. Five different channel models, namely, multiple access, broadcast, interference, relay and cognitive radio channels, are considered. In particular, throughput regions of multiple-access fading channels are characterized when multiple users, experiencing random data arrivals, transmit to a common receiver under statistical QoS constraints. Throughput regions of fading broadcast channels with random data arrivals in the presence of QoS requirements are studied when power control is employed at the transmitter. It is assumed that superposition coding with power control is performed at the transmitter with interference cancellation at the receivers. Optimal power control policies that maximize the weighted combination of the average arrival rates are investigated in the two-user case. Energy efficiency in two-user fading interference channels is studied when the transmitters are operating subject to QoS constraints. Specifically, energy efficiency is characterized by determining the corresponding minimum energy per bit requirements and wideband slope regions. Furthermore, transmission over a half-duplex relay channel with secrecy and QoS constraints is studied. Secrecy throughput is derived for the half duplex two-hop fading relay system operating in the presence of an eavesdropper. Fundamental limits on the energy efficiency of cognitive radio transmissions are analyzed in the presence of statistical quality of service (QoS) constraints. Minimum energy per bit and wideband slope expressions are obtained in order to identify the performance limits in terms of energy efficiency

Fast Reinforcement Learning for Energy-Efficient Wireless Communications

We consider the problem of energy-efficient point-to-point transmission of delay-sensitive data (e.g. multimedia data) over a fading channel. Existing research on this topic utilizes either physical-layer centric solutions, namely power-control and adaptive modulation and coding (AMC), or system-level solutions based on dynamic power management (DPM); however, there is currently no rigorous and unified framework for simultaneously utilizing both physical-layer centric and system-level techniques to achieve the minimum possible energy consumption, under delay constraints, in the presence of stochastic and a priori unknown traffic and channel conditions. In this report, we propose such a framework. We formulate the stochastic optimization problem as a Markov decision process (MDP) and solve it online using reinforcement learning. The advantages of the proposed online method are that (i) it does not require a priori knowledge of the traffic arrival and channel statistics to determine the jointly optimal power-control, AMC, and DPM policies; (ii) it exploits partial information about the system so that less information needs to be learned than when using conventional reinforcement learning algorithms; and (iii) it obviates the need for action exploration, which severely limits the adaptation speed and run-time performance of conventional reinforcement learning algorithms. Our results show that the proposed learning algorithms can converge up to two orders of magnitude faster than a state-of-the-art learning algorithm for physical layer power-control and up to three orders of magnitude faster than conventional reinforcement learning algorithms