Q-learning Channel Access Methods for Wireless Powered Internet of Things Networks

The Internet of Things (IoT) is becoming critical in our daily life. A key technology of interest in this thesis is Radio Frequency (RF) charging. The ability to charge devices wirelessly creates so called RF-energy harvesting IoT networks. In particular, there is a hybrid access point (HAP) that provides energy in an on-demand manner to RF-energy harvesting devices. These devices then collect data and transmit it to the HAP. In this respect, a key issue is ensuring devices have a high number of successful transmissions. There are a number of issues to consider when scheduling the transmissions of devices in the said network. First, the channel gain to/from devices varies over time. This means the efficiency to deliver energy to devices and to transmit the same amount of data is different over time. Second, during channel access, devices are not aware of the energy level of other devices nor whether they will transmit data. Third, devices have non-causal knowledge of their energy arrivals and channel gain information. Consequently, they do not know whether they should delay their transmissions in hope of better channel conditions or less contention in future time slots or doing so would result in energy overflow

Energy harvesting-aware design of wireless networks

Recent advances in low-power electronics and energy-harvesting (EH) technologies enable the design of self-sustained devices that collect part, or all, of the needed energy from the environment. Several systems can take advantage of EH, ranging from portable devices to wireless sensor networks (WSNs). While conventional design for battery-powered systems is mainly concerned with the battery lifetime, a key advantage of EH is that it enables potential perpetual operation of the devices, without requiring maintenance for battery substitutions. However, the inherent unpredictability regarding the amount of energy that can be collected from the environment might cause temporary energy shortages, which might prevent the devices to operate regularly. This uncertainty calls for the development of energy management techniques that are tailored to the EH dynamics. While most previous work on EH-capable systems has focused on energy management for single devices, the main contributions of this dissertation is the analysis and design of medium access control (MAC) protocols for WSNs operated by EH-capable devices. In particular, the dissertation first considers random access MAC protocols for single-hop EH networks, in which a fusion center collects data from a set of nodes distributed in its surrounding. MAC protocols commonly used in WSNs, such as time division multiple access (TDMA), framed-ALOHA (FA) and dynamic-FA (DFA) are investigated in the presence of EH-capable devices. A new ALOHA-based MAC protocol tailored to EH-networks, referred to as energy group-DFA (EG-DFA), is then proposed. In EG-DFA nodes with similar energy availability are grouped together and access the channel independently from other groups. It is shown that EG-DFA significantly outperforms the DFA protocol. Centralized scheduling-based MAC protocols for single-hop EH-networks with communication resource constraints are considered next. Two main scenarios are addressed, namely: i) nodes exclusively powered via EH; ii) nodes powered by a hybrid energy storage system, which is composed by a non-rechargeable battery and a capacitor charged via EH. For the former case the goal is the maximization of the network throughput, while in the latter the aim is maximizing the lifetime of the non-rechargeable batteries. For both scenarios optimal scheduling policies are derived by assuming different levels of information available at the fusion center about the energy availability at the nodes. When optimal policies are not derived explicitly, suboptimal policies are proposed and compared with performance upper bounds. Energy management policies for single devices have been investigated as well by focusing on radio frequency identification (RFID) systems, when the latter are operated by enhanced RFID tags with energy harvesting capabilities

Stability of synchronous queued RFID networks

Queued Radio Frequency Identification (RFID) networks arise naturally in many applications, where tags are grouped into batches, and each batch must be processed before the next reading job starts. In these cases, the system must be able to handle all incoming jobs, keeping the queue backlogs bounded. This property is called stability. Besides, in RFID networks, it is common that some readers cannot operate at the same time, due to mutual interferences. This fact reduces the maximum traffic that readers can process since they have to share the channel. Synchronous networks share the channel using a TDMA approach. The goal of this work is to analytically determine whether a synchronous queued RFID network attains stable operation under a given incoming traffic. Stability depends on the service rate, which is characterized in this paper using an exact numerical method based on a recursive analytical approach, overcoming the limitations of previous works, which were based on simplifications. We also address different flow optimization problems, such as computing the maximum joint traffic that a network can process stably, selecting the minimal number of readers to process a given total load, or determining the optimal timeslot duration, which are novel in the RFID literature.This work was supported by the Project AIM, (AEI/FEDER, EU) under Grant TEC2016-76465-C2-1-R

Stability of synchronous queued RFID networks

Queued Radio Frequency Identification (RFID) networks arise naturally in many applications, where tags are grouped into batches, and each batch must be processed before the next reading job starts. In these cases, the system must be able to handle all incoming jobs, keeping the queue backlogs bounded. This property is called stability. Besides, in RFID networks, it is common that some readers cannot operate at the same time, due to mutual interferences. This fact reduces the maximum traffic that readers can process since they have to share the channel. Synchronous networks share the channel using a TDMA approach. The goal of this work is to analytically determine whether a synchronous queued RFID network attains stable operation under a given incoming traffic. Stability depends on the service rate, which is characterized in this paper using an exact numerical method based on a recursive analytical approach, overcoming the limitations of previous works, which were based on simplifications. We also address different flow optimization problems, such as computing the maximum joint traffic that a network can process stably, selecting the minimal number of readers to process a given total load, or determining the optimal timeslot duration, which are novel in the RFID literature.Ministerio de Economía, Industria y Competitividad | Ref. TEC2016-76465-C2-1-

Age of Information in Slotted ALOHA With Energy Harvesting

We examine the age of information (AoI) in a status update system that incorporates energy harvesting and the slotted ALOHA protocol. We derive analytically the average AoI and the probability that the AoI exceeds a given threshold. Via numerical results, we investigate the trade-off between i) transmitting a new update whenever possible, and ii) transmitting only when sufficient energy is available to increase the chance of successful delivery. The two strategies turn out to be beneficial for low and high update generation rates, respectively. However, an optimized approach that balances the two strategies outperforms them significantly in terms of both AoI and throughput