Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Load-balancing rendezvous approach for mobility-enabled adaptive energy-efficient data collection in WSNs

Copyright © 2020 KSII The tradeoff between energy conservation and traffic balancing is a dilemma problem in Wireless Sensor Networks (WSNs). By analyzing the intrinsic relationship between cluster properties and long distance transmission energy consumption, we characterize three node sets of the cluster as a theoretical foundation to enhance high performance of WSNs, and propose optimal solutions by introducing rendezvous and Mobile Elements (MEs) to optimize energy consumption for prolonging the lifetime of WSNs. First, we exploit an approximate method based on the transmission distance from the different node to an ME to select suboptimal Rendezvous Point (RP) on the trajectory for ME to collect data. Then, we define data transmission routing sequence and model rendezvous planning for the cluster. In order to achieve optimization of energy consumption, we specifically apply the economic theory called Diminishing Marginal Utility Rule (DMUR) and create the utility function with regard to energy to develop an adaptive energy consumption optimization framework to achieve energy efficiency for data collection. At last, Rendezvous Transmission Algorithm (RTA) is proposed to better tradeoff between energy conservation and traffic balancing. Furthermore, via collaborations among multiple MEs, we design Two-Orbit Back-Propagation Algorithm (TOBPA) which concurrently handles load imbalance phenomenon to improve the efficiency of data collection. The simulation results show that our solutions can improve energy efficiency of the whole network and reduce the energy consumption of sensor nodes, which in turn prolong the lifetime of WSNs

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier

Energy Harvesting Wireless Communications: A Review of Recent Advances

This article summarizes recent contributions in the broad area of energy harvesting wireless communications. In particular, we provide the current state of the art for wireless networks composed of energy harvesting nodes, starting from the information-theoretic performance limits to transmission scheduling policies and resource allocation, medium access and networking issues. The emerging related area of energy transfer for self-sustaining energy harvesting wireless networks is considered in detail covering both energy cooperation aspects and simultaneous energy and information transfer. Various potential models with energy harvesting nodes at different network scales are reviewed as well as models for energy consumption at the nodes.Comment: To appear in the IEEE Journal of Selected Areas in Communications (Special Issue: Wireless Communications Powered by Energy Harvesting and Wireless Energy Transfer

DESIGN OF RELIABLE AND SUSTAINABLE WIRELESS SENSOR NETWORKS: CHALLENGES, PROTOCOLS AND CASE STUDIES

Integrated with the function of sensing, processing, and wireless communication, wireless sensors are attracting strong interest for a variety of monitoring and control applications. Wireless sensor networks (WSNs) have been deployed for industrial and remote monitoring purposes. As energy shortage is a worldwide problem, more attention has been placed on incorporating energy harvesting devices in WSNs. The main objective of this research is to systematically study the design principles and technical approaches to address three key challenges in designing reliable and sustainable WSNs; namely, communication reliability, operation with extremely low and dynamic power sources, and multi-tier network architecture. Mathematical throughput models, sustainable WSN communication strategies, and multi-tier network architecture are studied in this research to address these challenges, leading to protocols for reliable communication, energy-efficient operation, and network planning for specific application requirements. To account for realistic operating conditions, the study has implemented three distinct WSN testbeds: a WSN attached to the high-speed rotating spindle of a turning lathe, a WSN powered by a microbial fuel cell based energy harvesting system, and a WSN with a multi-tier network architecture. With each testbed, models and protocols are extracted, verified and analyzed. Extensive research has studied low power WSNs and energy harvesting capabilities. Despite these efforts, some important questions have not been well understood. This dissertation addresses the following three dimensions of the challenge. First, for reliable communication protocol design, mathematical throughput or energy efficiency estimation models are essential, yet have not been investigated accounting for specific application environment characteristics and requirements. Second, for WSNs with energy harvesting power sources, most current networking protocols do not work efficiently with the systems considered in this dissertation, such as those powered by extremely low and dynamic energy sources. Third, for multi-tier wireless network system design, routing protocols that are adaptive to real-world network conditions have not been studied. This dissertation focuses on these questions and explores experimentally derived mathematical models for designing protocols to meet specific application requirements. The main contributions of this research are 1) for industrial wireless sensor systems with fast-changing but repetitive mobile conditions, understand the performance and optimal choice of reliable wireless sensor data transmission methods, 2) for ultra-low energy harvesting wireless sensor devices, design an energy neutral communication protocol, and 3) for distributed rural wireless sensor systems, understand the efficiency of realistic routing in a multi-tier wireless network. Altogether, knowledge derived from study of the systems, models, and protocols in this work fuels the establishment of a useful framework for designing future WSNs

QoS-Aware Energy Management and Node Scheduling Schemes for Sensor Network-Based Surveillance Applications

Recent advances in wireless technologies have led to an increased deployment of Wireless Sensor Networks (WSNs) for a plethora of diverse surveillance applications such as health, military, and environmental. However, sensor nodes in WSNs usually suffer from short device lifetime due to severe energy constraints and therefore, cannot guarantee to meet the Quality of Service (QoS) needs of various applications. This is proving to be a major hindrance to the widespread adoption of WSNs for such applications. Therefore, to extend the lifetime of WSNs, it is critical to optimize the energy usage in sensor nodes that are often deployed in remote and hostile terrains. To this effect, several energy management schemes have been proposed recently. Node scheduling is one such strategy that can prolong the lifetime of WSNs and also helps to balance the workload among the sensor nodes. In this article, we discuss on the energy management techniques of WSN with a particular emphasis on node scheduling and propose an energy management life-cycle model and an energy conservation pyramid to extend the network lifetime of WSNs. We have provided a detailed classification and evaluation of various node scheduling schemes in terms of their ability to fulfill essential QoS requirements, namely coverage, connectivity, fault tolerance, and security. We considered essential design issues such as network type, deployment pattern, sensing model in the classification process. Furthermore, we have discussed the operational characteristics of schemes with their related merits and demerits. We have compared the efficacy of a few well known graph-based scheduling schemes with suitable performance analysis graph. Finally, we study challenges in designing and implementing node scheduling schemes from a QoS perspective and outline open research problems

Quality-Aware Scheduling Algorithms in Renewable Sensor

Wireless sensor network has emerged as a key technology for various applications such as environmental sensing, structural health monitoring, and area surveillance. Energy is by far one of the most critical design hurdles that hinders the deployment of wireless sensor networks. The lifetime of traditional battery-powered sensor networks is limited by the capacities of batteries. Even many energy conservation schemes were proposed to address this constraint, the network lifetime is still inherently restrained, as the consumed energy cannot be replenished easily. Fully addressing this issue requires energy to be replenished quite often in sensor networks (renewable sensor networks). One viable solution to energy shortages is enabling each sensor to harvest renewable energy from its surroundings such as solar energy, wind energy, and so on. In comparison with their conventional counterparts, the network lifetime in renewable sensor networks is no longer a main issue, since sensors can be recharged repeatedly. This results in a research focus shift from the network lifetime maximization in traditional sensor networks to the network performance optimization (e.g., monitoring quality). This thesis focuses on these issues and tackles important problems in renewable sensor networks as follows. We first study the target coverage optimization in renewable sensor networks via sensor duty cycle scheduling, where a renewable sensor network consisting of a set of heterogeneous sensors and a stationary base station need to be scheduled to monitor a set of targets in a monitoring area (e.g., some critical facilities) for a specified period, by transmitting their sensing data to the base station through multihop relays in a real-time manner. We formulate a coverage maximization problem in a renewable sensor network which is to schedule sensor activities such that the monitoring quality is maximized, subject to that the communication network induced by the activated sensors and the base station at each time moment is connected. We approach the problem for a given monitoring period by adopting a general strategy. That is, we divide the entire monitoring period into equal numbers of time slots and perform sensor activation or inactivation scheduling in the beginning of each time slot. As the problem is NP-hard, we devise efficient offline centralized and distributed algorithms for it, provided that the amount of harvested energy of each sensor for a given monitoring period can be predicted accurately. Otherwise, we propose an online adaptive framework to handle energy prediction fluctuation for this monitoring period. We conduct extensive experiments, and the experimental results show that the proposed solutions are very promising. We then investigate the data collection optimization in renewable sensor networks by exploiting sink mobility, where a mobile sink travels around the sensing field to collect data from sensors through one-hop transmission. With one-hop transmission, each sensor could send data directly to the mobile sink without any relay, and thus no energy are consumed on forwarding packets for others which is more energy efficient in comparison with multi-hop relays. Moreover, one-hop transmission particularly is very useful for a disconnected network, which may be due to the error-prone nature of wireless communication or the physical limit (e.g., some sensors are physically isolated), while multi-hop transmission is not applicable. In particular, we investigate two different kinds of mobile sinks, and formulate optimization problems under different scenarios, for which both centralized and distributed solutions are proposed accordingly. We study the performance of the proposed solutions and validate their effectiveness in improving the data quality. Since the energy harvested often varies over time, we also consider the scenario of renewable sensor networks by utilizing wireless energy transfer technology, where a mobile charging vehicle periodically travels inside the sensing field and charges sensors without any plugs or wires. Specifically, we propose a novel charging paradigm and formulate an optimization problem with an objective of maximizing the number of sensors charged per tour. We devise an offline approximation algorithm which runs in quasi-polynomial time and develop efficient online sensor charging algorithms, by considering the dynamic behaviors of sensors’ various sensing and transmission activities. To study the efficiency of the proposed algorithms, we conduct extensive experiments and the experimental results demonstrate that the proposed algorithms are very efficient. We finally conclude our work and discuss potential research topics which derive from the studies of this thesis

Data Collection in Two-Tier IoT Networks with Radio Frequency (RF) Energy Harvesting Devices and Tags

The Internet of things (IoT) is expected to connect physical objects and end-users using technologies such as wireless sensor networks and radio frequency identification (RFID). In addition, it will employ a wireless multi-hop backhaul to transfer data collected by a myriad of devices to users or applications such as digital twins operating in a Metaverse. A critical issue is that the number of packets collected and transferred to the Internet is bounded by limited network resources such as bandwidth and energy. In this respect, IoT networks have adopted technologies such as time division multiple access (TDMA), signal interference cancellation (SIC) and multiple-input multiple-output (MIMO) in order to increase network capacity. Another fundamental issue is energy. To this end, researchers have exploited radio frequency (RF) energy-harvesting technologies to prolong the lifetime of energy constrained sensors and smart devices. Specifically, devices with RF energy harvesting capabilities can rely on ambient RF sources such as access points, television towers, and base stations. Further, an operator may deploy dedicated power beacons that serve as RF-energy sources. Apart from that, in order to reduce energy consumption, devices can adopt ambient backscattering communication technologies. Advantageously, backscattering allows devices to communicate using negligible amount of energy by modulating ambient RF signals. To address the aforementioned issues, this thesis first considers data collection in a two-tier MIMO ambient RF energy-harvesting network. The first tier consists of routers with MIMO capability and a set of source-destination pairs/flows. The second tier consists of energy harvesting devices that rely on RF transmissions from routers for energy supply. The problem is to determine a minimum-length TDMA link schedule that satisfies the traffic demand of source-destination pairs and energy demand of energy harvesting devices. It formulates the problem as a linear program (LP), and outlines a heuristic to construct transmission sets that are then used by the said LP. In addition, it outlines a new routing metric that considers the energy demand of energy harvesting devices to cope with routing requirements of IoT networks. The simulation results show that the proposed algorithm on average achieves 31.25% shorter schedules as compared to competing schemes. In addition, the said routing metric results in link schedules that are at most 24.75% longer than those computed by the LP

Integrated Data and Energy Communication Network: A Comprehensive Survey

OAPA In order to satisfy the power thirsty of communication devices in the imminent 5G era, wireless charging techniques have attracted much attention both from the academic and industrial communities. Although the inductive coupling and magnetic resonance based charging techniques are indeed capable of supplying energy in a wireless manner, they tend to restrict the freedom of movement. By contrast, RF signals are capable of supplying energy over distances, which are gradually inclining closer to our ultimate goal – charging anytime and anywhere. Furthermore, transmitters capable of emitting RF signals have been widely deployed, such as TV towers, cellular base stations and Wi-Fi access points. This communication infrastructure may indeed be employed also for wireless energy transfer (WET). Therefore, no extra investment in dedicated WET infrastructure is required. However, allowing RF signal based WET may impair the wireless information transfer (WIT) operating in the same spectrum. Hence, it is crucial to coordinate and balance WET and WIT for simultaneous wireless information and power transfer (SWIPT), which evolves to Integrated Data and Energy communication Networks (IDENs). To this end, a ubiquitous IDEN architecture is introduced by summarising its natural heterogeneity and by synthesising a diverse range of integrated WET and WIT scenarios. Then the inherent relationship between WET and WIT is revealed from an information theoretical perspective, which is followed by the critical appraisal of the hardware enabling techniques extracting energy from RF signals. Furthermore, the transceiver design, resource allocation and user scheduling as well as networking aspects are elaborated on. In a nutshell, this treatise can be used as a handbook for researchers and engineers, who are interested in enriching their knowledge base of IDENs and in putting this vision into practice