Energy Harvesting Communication System with SOC-Dependent Energy Storage Losses

The popularity of Energy Harvesting Devices (EHDs) has grown in the past few years, thanks to their capability of prolonging the network lifetime. In reality, EHDs are affected by several inefficiencies, e.g., energy leakage, battery degradation or storage losses. In this work we consider an energy harvesting transmitter with storage inefficiencies. In particular, we assume that when new energy has to be stored in the battery, part of this is wasted and the losses depend upon the current state of charge of the device. This is a practical realistic assumption, e.g., for a capacitor, that changes the structure of the optimal transmission policy. We analyze the throughput maximization problem with a dynamic programming approach and prove that, given the battery status and the channel gain, the optimal transmission policy is deterministic. We derive numerical results for the energy losses in a capacitor and show the presence of a \emph{loop effect} that degrades the system performance if the optimal policy is not considered.Comment: In Proc. IEEE Twelfth Int. Symposium on Wireless Communication Systems (ISWCS), pp. 406-410, Aug. 201

On the Effects of Battery Imperfections in an Energy Harvesting Device

Energy Harvesting allows the devices in a Wireless Sensor Network to recharge their batteries through environmental energy sources. While in the literature the main focus is on devices with ideal batteries, in reality several inefficiencies have to be considered to correctly design the operating regimes of an Energy Harvesting Device (EHD). In this work we describe how the throughput optimization problem changes under \emph{real battery} constraints in an EHD. In particular, we consider imperfect knowledge of the state of charge of the battery and storage inefficiencies, \emph{i.e.}, part of the harvested energy is wasted in the battery recharging process. We formulate the problem as a Markov Decision Process, basing our model on some realistic observations about transmission, consumption and harvesting power. We find the performance upper bound with a real battery and numerically discuss the novelty introduced by the real battery effects. We show that using the \emph{old} policies obtained without considering the real battery effects is strongly sub-optimal and may even result in zero throughput.Comment: In Proc. IEEE International Conference on Computing, Networking and Communications, pp. 942-948, Feb. 201

Towards Self-Control of Service Rate for Battery Management in Energy Harvesting Devices

We consider the operation of an energy harvesting wireless device (sensor node) powered by a rechargeable battery, taking non-idealities into account. In particular, we consider sudden decrease and increase of the battery level (leakage and charge recovery consequently) due to the inner diffusion processes in the battery. These processes are affecting the stability of the device operation. In particular, leakage accelerates the depletion of the battery, which results in inactive periods of the device and, thus, potential data loss. In this paper, we propose a simplified self-control management of a battery expressed by restrictions, which could be used for an efficient operational strategy of the device. To achieve this, we rely on the double-queue model which includes the imperfections of the battery operation and bi-dimensional battery value. This includes both apparent, i.e., available at the electrodes and true energy levels of a battery. These levels can be significantly different because of deep discharge events and can equalize thanks to charge recovery effect. We performed some simulation and observed that we can diminish the models variable number to predict possible unwanted events such as apparent discharge events (when the areas near electrodes are depleted while other areas of the battery still contain some energy) and data losses. This observation helps to achieve sufficiently effective self-control management by knowing and managing just few parameters, and therefore offers valuable directions for the development of autonomic and self-sustainable operation

Joint Transmission and Energy Transfer Policies for Energy Harvesting Devices with Finite Batteries

One of the main concerns in traditional Wireless Sensor Networks (WSNs) is energy efficiency. In this work, we analyze two techniques that can extend network lifetime. The first is Ambient \emph{Energy Harvesting} (EH), i.e., the capability of the devices to gather energy from the environment, whereas the second is Wireless \emph{Energy Transfer} (ET), that can be used to exchange energy among devices. We study the combination of these techniques, showing that they can be used jointly to improve the system performance. We consider a transmitter-receiver pair, showing how the ET improvement depends upon the statistics of the energy arrivals and the energy consumption of the devices. With the aim of maximizing a reward function, e.g., the average transmission rate, we find performance upper bounds with and without ET, define both online and offline optimization problems, and present results based on realistic energy arrivals in indoor and outdoor environments. We show that ET can significantly improve the system performance even when a sizable fraction of the transmitted energy is wasted and that, in some scenarios, the online approach can obtain close to optimal performance.Comment: 16 pages, 12 figure

Energy harvesting communication system with SOC-dependent energy storage losses

The popularity of Energy Harvesting Devices (EHDs) has grown in the past few years, thanks to their capability of prolonging the network lifetime. In reality, EHDs are affected by several inefficiencies, e.g., energy leakage, battery degradation or storage losses. In this work we consider an energy harvesting transmitter with storage inefficiencies. In particular, we assume that when new energy has to be stored in the battery, part of this is wasted and the losses depend upon the current state of charge of the device. This is a practical realistic assumption, e.g., for a capacitor, that changes the structure of the optimal transmission policy. We analyze the throughput maximization problem with a dynamic programming approach and prove that, given the battery status and the channel gain, the optimal transmission policy is deterministic. We derive numerical results for the energy losses in a capacitor and show the presence of a loop effect that degrades the system performance if the optimal policy is not considered

Optimization and Learning Approaches for Energy Harvesting Wireless Communication Systems

Emerging technologies such as Internet of Things (IoT) and Industry 4.0 are now possible thanks to the advances in wireless sensor networks. In such applications, the wireless communication nodes play a key role because they provide the connection between different sensors as well as the communication to the outside world. In general, these wireless communication nodes are battery operated. However, depending on the specific application, charging or replacing the batteries can be too expensive or even infeasible, e.g., when the nodes are located in remote locations or inside structures. Therefore, in order to provide sustainable service and to reduce the operation expenses, energy harvesting (EH) has been considered as a promising technology in which the nodes collect energy from the environment using natural or man-made energy sources such as solar or electromagnetic radiation. The idea behind EH is that the wireless communication nodes can recharge their batteries while in idle mode or while transmitting data to neighboring nodes. As a result, the lifetime of the wireless communication network is not limited by the availability of energy. The consideration of EH brings new challenges in the design of transmission policies. This is because in addition to the fluctuating channel conditions and data arrival processes, the variability of the amount of energy available for the communication should be taken into account. Moreover, the three processes, EH, data arrival and channel fading, should be jointly considered in order to achieve optimum performance. In this context, this dissertation contributes to the research on EH wireless communication networks by considering power allocation and resource allocation problems in four different scenarios, namely, EH point-to-point, EH two-hop, EH broadcast and EH multiple access, which are the fundamental constituents of more complicated networks. Specifically, we determine the optimal allocation policies and the corresponding upper bounds of the achievable performance by considering offline approaches in which non-causal knowledge regarding system dynamics, i.e., the EH, data arrival and channel fading processes, is assumed. Furthermore, we overcome this unrealistic assumption by developing novel learning approaches, based on reinforcement learning, under the practical assumption that only causal knowledge of the system dynamics is available. First, we focus on the EH point-to-point scenario where an EH transmitter sends data to a receiver. For this scenario, we formulate the power allocation problem for throughput maximization considering not only the transmit power, but also the energy consumed by the circuit. Adopting an offline approach, we characterize the optimum power allocation policy and exploit this analysis in the development of a learning approach. Specifically, we develop a novel learning algorithm which considers a realistic EH point-to-point scenario, i.e., only causal knowledge of the system dynamics is assumed to be available. For the proposed learning algorithm, we exploit linear function approximation to cope with the infinite number of values the harvested energy, the incoming data and the channel coefficients can take. In particular, we propose four feature functions which are inspired by the characteristics of the problem and the insights gained from the offline approach. Through numerical simulations, we show that the proposed learning approach achieves a performance close to the offline optimum without the requirement of non-causal knowledge of the system dynamics. Moreover, it can achieve a performance up to 50% higher than the performance of reference learning schemes such as Q-learning, which do not exploit the characteristics of the problem. Secondly, we investigate an EH two-hop scenario in which an EH transmitter communicates with a receiver via an EH relay. For this purpose, we consider the main relaying strategies, namely, decode-and-forward and amplify-and-forward. Furthermore, we consider both, the transmit power and the energy consumed by the circuit in each of the EH nodes. For the EH decode-and-forward relay, we formulate the power allocation problem for throughput maximization and consider an offline approach to find the optimum power allocation policy. We show that the optimal power allocation policies of both nodes, transmitter and relay, depend on each other. Additionally, following a learning approach, we investigate a more realistic scenario in which the EH transmitter and the EH decode-and-forward relay have only partial and causal knowledge about the system dynamics, i.e., each node has only causal knowledge about the EH, data arrival and channel fading processes associated to it. To this aim, two novel learning algorithms are proposed which take into account whether or not the EH nodes cooperate with each other to improve their learning processes. For the cooperative case, we propose the inclusion of a signaling phase in which the EH nodes exchange their current parameters. Through numerical simulations, we show that by providing the nodes with a complete view of the system state in a signaling phase, a performance gain of up to 40% can be achieved compared to the case when no cooperation is considered. Following a similar procedure, we investigate the EH two-hop scenario with an EH amplify-and-forward relay. We show that the resulting power allocation problem for throughput maximization is non-convex. Consequently, we propose an offline approach based on a branch-and-bound algorithm tailored to the EH two-hop scenario to find the optimal power allocation policy. Additionally, a centralized learning algorithm is proposed for the realistic case in which only causal knowledge of the system dynamics is available. The proposed learning approach exploits the fact that, with an amplify-and-forward relay, the communication between the transmitter and the receiver depends on a single effective channel, which is composed of the link between the transmitter and the relay, the relay gain and the channel from the relay to the receiver. By means of numerical simulations, we show that the proposed learning algorithm achieves a performance up to two times higher than the performance achieved by reference schemes. Additionally, the extension of the proposed approaches to EH multi-hop scenarios is discussed. Thirdly, an EH broadcast scenario in which an EH transmitter sends individual data to multiple receivers is studied. We show that the power allocation problem for throughput maximization in this scenario leads to a non-convex problem when an arbitrary number of receivers is considered. However, following an offline approach we find the optimal power allocation policy for the special case when two receivers are considered. Furthermore, inspired by the offline approach for two users, a novel learning approach which does not pose any restriction on the number of receiver nodes is developed. The proposed learning approach is a two-stage learning algorithm which separates the learning task into two subtasks: determining how much power to use in each time interval and deciding how to split this selected power for the transmission of the individual data intended for each receiver. Through numerical simulations, we show that the separation of tasks leads to a performance up to 40% higher than the one achieved by standard learning techniques, specially for large numbers of receivers. Finally, an EH multiple access scenario is considered in which multiple EH transmitters communicate with a single receiver using multiple orthogonal resources. In this case, the focus is on the formulation of the resource allocation problem considering the EH processes at the different transmitters. We show that the resulting resource allocation problem falls into the category of non-linear knapsack problems which are known to be NP-hard. Therefore, we propose an offline approach based on dynamic programming to find the optimal solution. Furthermore, by exploiting the characteristics of the scenario, a novel learning approach is proposed which breaks the original resource allocation problem into smaller subproblems. As a result, it is able to handle the exponential growth of the space of possible solutions when the network size increases. Through numerical simulations, we show that in contrast to conventional reinforcement learning algorithms, the proposed learning approach is able to find the resource allocation policy that aims at maximizing the throughput when the network size is large. Furthermore, it achieves a performance up to 25% higher than the performance of the greedy policy that allocates the resources to the users with the best channel conditions. Additionally, in order to carry out a full assessment of the proposed learning algorithms, we provide convergence guarantees and a computational complexity analysis for all the developed learning approaches in the four considered scenarios

Stochastic Optimization of Energy Harvesting Wireless Communication Networks

Energy harvesting from environmental energy sources (e.g., sunlight) or from man-made sources (e.g., RF energy) has been a game-changing paradigm, which enabled the possibility of making the devices in the Internet of Things or wireless sensor networks operate autonomously and with high performance for years or even decades without human intervention. However, an energy harvesting system must be correctly designed to achieve such a goal and therefore the energy management problem has arisen and become a critical aspect to consider in modern wireless networks. In particular, in addition to the hardware (e.g., in terms of circuitry design) and application point of views (e.g., sensor deployment), also the communication protocol perspective must be explicitly taken into account; indeed, the use of the wireless communication interface may play a dominant role in the energy consumption of the devices, and thus must be correctly designed and optimized. This analysis represents the focus of this thesis. Energy harvesting for wireless system has been a very active research topic in the past decade. However, there are still many aspects that have been neglected or not completely analyzed in the literature so far. Our goal is to address and solve some of these new problems using a common stochastic optimization setup based on dynamic programming. In particular, we formulate both the centralized and decentralized optimization problems in an energy harvesting network with multiple devices, and discuss the interrelations between these two schemes; we study the combination of environmental energy harvesting and wireless energy transfer to improve the transmission rate of the network and achieve a balanced situation; we investigate the long-term optimization problem in wireless powered communication networks, in which the receiver supplies wireless energy to the terminal nodes; we deal with the energy storage inefficiencies of the energy harvesting devices, and show that traditional policies may be strongly suboptimal in this context; finally, we investigate how it is possible to increase secrecy in a wireless link where a third malicious party eavesdrops the information transmitted by an energy harvesting node