Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Optimal Data Scheduling and Admission Control for Backscatter Sensor Networks

© 2017 IEEE. This paper studies the data scheduling and admission control problem for a backscatter sensor network (BSN). In the network, instead of initiating their own transmissions, the sensors can send their data to the gateway just by switching their antenna impedance and reflecting the received RF signals. As such, we can reduce remarkably the complexity, the power consumption, and the implementation cost of sensor nodes. Different sensors may have different functions, and data collected from each sensor may also have a different status, e.g., urgent or normal, and thus we need to take these factors into account. Therefore, in this paper, we first introduce a system model together with a mechanism in order to address the data collection and scheduling problem in the BSN. We then propose an optimization solution using the Markov decision process framework and a reinforcement learning algorithm based on the linear function approximation method, with the aim of finding the optimal data collection policy for the gateway. Through simulation results, we not only show the efficiency of the proposed solution compared with other baseline policies, but also present the analysis for data admission control policy under different classes of sensors as well as different types of data

RF Energy Harvesting Wireless Communication: RF Environment, Device Hardware and Practical Issues

Radio frequency (RF) based wireless power transfer provides an attractive solution to extend the lifetime of power-constrained wireless sensor networks. Through harvesting RF energy from surrounding environments or dedicated energy sources, low-power wireless devices can be self-sustaining and environment-friendly. These features make the RF energy harvesting wireless communication (RF-EHWC) technique attractive to a wide range of applications. The objective of this article is to investigate the latest research activities on the practical RF-EHWC design. The distribution of RF energy in the real environment, the hardware design of RF-EHWC devices and the practical issues in the implementation of RF-EHWC networks are discussed. At the end of this article, we introduce several interesting applications that exploit the RF-EHWC technology to provide smart healthcare services for animals, wirelessly charge the wearable devices, and implement 5G-assisted RF-EHWC

Data Collection in Two-Tier IoT Networks with Radio Frequency (RF) Energy Harvesting Devices and Tags

The Internet of things (IoT) is expected to connect physical objects and end-users using technologies such as wireless sensor networks and radio frequency identification (RFID). In addition, it will employ a wireless multi-hop backhaul to transfer data collected by a myriad of devices to users or applications such as digital twins operating in a Metaverse. A critical issue is that the number of packets collected and transferred to the Internet is bounded by limited network resources such as bandwidth and energy. In this respect, IoT networks have adopted technologies such as time division multiple access (TDMA), signal interference cancellation (SIC) and multiple-input multiple-output (MIMO) in order to increase network capacity. Another fundamental issue is energy. To this end, researchers have exploited radio frequency (RF) energy-harvesting technologies to prolong the lifetime of energy constrained sensors and smart devices. Specifically, devices with RF energy harvesting capabilities can rely on ambient RF sources such as access points, television towers, and base stations. Further, an operator may deploy dedicated power beacons that serve as RF-energy sources. Apart from that, in order to reduce energy consumption, devices can adopt ambient backscattering communication technologies. Advantageously, backscattering allows devices to communicate using negligible amount of energy by modulating ambient RF signals. To address the aforementioned issues, this thesis first considers data collection in a two-tier MIMO ambient RF energy-harvesting network. The first tier consists of routers with MIMO capability and a set of source-destination pairs/flows. The second tier consists of energy harvesting devices that rely on RF transmissions from routers for energy supply. The problem is to determine a minimum-length TDMA link schedule that satisfies the traffic demand of source-destination pairs and energy demand of energy harvesting devices. It formulates the problem as a linear program (LP), and outlines a heuristic to construct transmission sets that are then used by the said LP. In addition, it outlines a new routing metric that considers the energy demand of energy harvesting devices to cope with routing requirements of IoT networks. The simulation results show that the proposed algorithm on average achieves 31.25% shorter schedules as compared to competing schemes. In addition, the said routing metric results in link schedules that are at most 24.75% longer than those computed by the LP

Mitigating the Event and Effect of Energy Holes in Multi-hop Wireless Sensor Networks Using an Ultra-Low Power Wake-up Receiver and an Energy Scheduling Technique

This research work presents an algorithm for extending network lifetime in multi-hop wireless sensor networks (WSN). WSNs face energy gap issues around sink nodes due to the transmission of large amounts of data through nearby sensor nodes. The limited power supply to the nodes limits the lifetime of the network, which makes energy efficiency crucial. Multi-hop communication has been proposed as an efficient strategy, but its power consumption remains a research challenge. In this study, an algorithm is developed to mitigate energy holes around the sink nodes by using a modified ultra-low-power wake-up receiver and an energy scheduling technique. Efficient power scheduling reduces the power consumption of the relay node, and when the residual power of the sensor node falls below a defined threshold, the power emitters charge the nodes to eliminate energy-hole problems. The modified wake-up receiver improves sensor sensitivity while staying within the micro-power budget. This study's simulations showed that the developed RF energy harvesting algorithm outperformed previous work, achieving a 30% improvement in average charged energy (AEC), a 0.41% improvement in average energy (AEH), an 8.39% improvement in the number of energy transmitters, an 8.59% improvement in throughput, and a 0.19 decrease in outage probability compared to the existing network lifetime enhancement of multi-hop wireless sensor networks by RF Energy Harvesting algorithm. Overall, the enhanced power efficiency technique significantly improves the performance of WSNs

Mobile Power Network for Ultimate Mobility without Battery Life Anxiety

Similar to the evolution from the wired Internet to mobile Internet (MI), the growing demand for power delivery anywhere and anytime appeals for power grid transformation from wired to mobile domain. We propose here the next generation of power delivery network -- mobile power network (MPN) for wireless power transfer within a mobile range from several meters to tens of meters. At first, we present the MPN's concept evolution and application scenarios. Then, we introduce the MPN's supporting technology, namely resonant beam charging (RBC). As a long-range wireless power transfer (WPT) method, RBC can safely deliver multi-Watt power to multiple devices concurrently. Meanwhile, the recent progress in RBC research has been summarized. Next, we specify the MPN's architecture to provide the wide-area WPT coverage. Finally, we discuss the MPN's features and challenges. MPN can enable the ultimate mobility by cutting the final cord of mobile devices, realizing the "last-mile" mobile power delivery.Comment: 10 pages, 5 figure