Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Energy harvesting-aware design of wireless networks

Recent advances in low-power electronics and energy-harvesting (EH) technologies enable the design of self-sustained devices that collect part, or all, of the needed energy from the environment. Several systems can take advantage of EH, ranging from portable devices to wireless sensor networks (WSNs). While conventional design for battery-powered systems is mainly concerned with the battery lifetime, a key advantage of EH is that it enables potential perpetual operation of the devices, without requiring maintenance for battery substitutions. However, the inherent unpredictability regarding the amount of energy that can be collected from the environment might cause temporary energy shortages, which might prevent the devices to operate regularly. This uncertainty calls for the development of energy management techniques that are tailored to the EH dynamics. While most previous work on EH-capable systems has focused on energy management for single devices, the main contributions of this dissertation is the analysis and design of medium access control (MAC) protocols for WSNs operated by EH-capable devices. In particular, the dissertation first considers random access MAC protocols for single-hop EH networks, in which a fusion center collects data from a set of nodes distributed in its surrounding. MAC protocols commonly used in WSNs, such as time division multiple access (TDMA), framed-ALOHA (FA) and dynamic-FA (DFA) are investigated in the presence of EH-capable devices. A new ALOHA-based MAC protocol tailored to EH-networks, referred to as energy group-DFA (EG-DFA), is then proposed. In EG-DFA nodes with similar energy availability are grouped together and access the channel independently from other groups. It is shown that EG-DFA significantly outperforms the DFA protocol. Centralized scheduling-based MAC protocols for single-hop EH-networks with communication resource constraints are considered next. Two main scenarios are addressed, namely: i) nodes exclusively powered via EH; ii) nodes powered by a hybrid energy storage system, which is composed by a non-rechargeable battery and a capacitor charged via EH. For the former case the goal is the maximization of the network throughput, while in the latter the aim is maximizing the lifetime of the non-rechargeable batteries. For both scenarios optimal scheduling policies are derived by assuming different levels of information available at the fusion center about the energy availability at the nodes. When optimal policies are not derived explicitly, suboptimal policies are proposed and compared with performance upper bounds. Energy management policies for single devices have been investigated as well by focusing on radio frequency identification (RFID) systems, when the latter are operated by enhanced RFID tags with energy harvesting capabilities

MODELING, METHODOLOGY AND APPLICATIONS FOR RESOURCE MANAGEMENT IN ENERGY HARVESTING SYSTEMS.

Ph.DDOCTOR OF PHILOSOPH

Joint Transmission and Energy Transfer Policies for Energy Harvesting Devices with Finite Batteries

One of the main concerns in traditional Wireless Sensor Networks (WSNs) is energy efficiency. In this work, we analyze two techniques that can extend network lifetime. The first is Ambient \emph{Energy Harvesting} (EH), i.e., the capability of the devices to gather energy from the environment, whereas the second is Wireless \emph{Energy Transfer} (ET), that can be used to exchange energy among devices. We study the combination of these techniques, showing that they can be used jointly to improve the system performance. We consider a transmitter-receiver pair, showing how the ET improvement depends upon the statistics of the energy arrivals and the energy consumption of the devices. With the aim of maximizing a reward function, e.g., the average transmission rate, we find performance upper bounds with and without ET, define both online and offline optimization problems, and present results based on realistic energy arrivals in indoor and outdoor environments. We show that ET can significantly improve the system performance even when a sizable fraction of the transmitted energy is wasted and that, in some scenarios, the online approach can obtain close to optimal performance.Comment: 16 pages, 12 figure

INTERMITTENTLY CONNECTED DELAY-TOLERANT WIRELESS SENSOR NETWORKS

Intermittently Connected Delay-Tolerant Wireless Sensor Networks (ICDT-WSNs), a branch of Wireless Sensor Networks (WSNs), have features of WSNs and the intermittent connectivity of Opportunistic Networks. The applications of ICDT-WSNs are increasing in recent years; however, the communication protocols suitable for this category of networks often fall short. Most of the existing communication protocols are designed for either WSNs or Opportunistic Networks with sufficient resources and tend to be inadequate for direct use in ICDT-WSNs. In this dissertation, we study ICDT-WSNs from the perspective of the characteristics, chal- lenges and possible solutions. A high-level overview of ICDT-WSNs is given, followed by a study of existing work and our solutions to address the problems of routing, flow control, error control, and storage management. The proposed solutions utilize the utility level of nodes and the connectedness of a network. In addition to the protocols for information transmissions to specific destinations, we also propose efficient mechanisms for information dissemination to arbitrary destinations. The study shows that our proposed solutions can achieve better performance than other state of the art communication protocols without sacrificing energy efficiency

Performance optimization of wireless sensor networks for remote monitoring

Wireless sensor networks (WSNs) have gained worldwide attention in recent years because of their great potential for a variety of applications such as hazardous environment exploration, military surveillance, habitat monitoring, seismic sensing, and so on. In this thesis we study the use of WSNs for remote monitoring, where a wireless sensor network is deployed in a remote region for sensing phenomena of interest while its data monitoring center is located in a metropolitan area that is geographically distant from the monitored region. This application scenario poses great challenges since such kind of monitoring is typically large scale and expected to be operational for a prolonged period without human involvement. Also, the long distance between the monitored region and the data monitoring center requires that the sensed data must be transferred by the employment of a third-party communication service, which incurs service costs. Existing methodologies for performance optimization of WSNs base on that both the sensor network and its data monitoring center are co-located, and therefore are no longer applicable to the remote monitoring scenario. Thus, developing new techniques and approaches for severely resource-constrained WSNs is desperately needed to maintain sustainable, unattended remote monitoring with low cost. Specifically, this thesis addresses the key issues and tackles problems in the deployment of WSNs for remote monitoring from the following aspects. To maximize the lifetime of large-scale monitoring, we deal with the energy consumption imbalance issue by exploring multiple sinks. We develop scalable algorithms which determine the optimal number of sinks needed and their locations, thereby dynamically identifying the energy bottlenecks and balancing the data relay workload throughout the network. We conduct experiments and the experimental results demonstrate that the proposed algorithms significantly prolong the network lifetime. To eliminate imbalance of energy consumption among sensor nodes, a complementary strategy is to introduce a mobile sink for data gathering. However, the limited communication time between the mobile sink and nodes results in that only part of sensed data will be collected and the rest will be lost, for which we propose the concept of monitoring quality with the exploration of sensed data correlation among nodes. We devise a heuristic for monitoring quality maximization, which schedules the sink to collect data from selected nodes, and uses the collected data to recover the missing ones. We study the performance of the proposed heuristic and validate its effectiveness in improving the monitoring quality. To strive for the fine trade-off between two performance metrics: throughput and cost, we investigate novel problems of minimizing cost with guaranteed throughput, and maximizing throughput with minimal cost. We develop approximation algorithms which find reliable data routing in the WSN and strategically balance workload on the sinks. We prove that the delivered solutions are fractional of the optimum. We finally conclude our work and discuss potential research topics which derive from the studies of this thesis

Coping with spectrum and energy scarcity in Wireless Networks: a Stochastic Optimization approach to Cognitive Radio and Energy Harvesting

In the last decades, we have witnessed an explosion of wireless communications and networking, spurring a great interest in the research community. The design of wireless networks is challenged by the scarcity of resources, especially spectrum and energy. In this thesis, we explore the potential offered by two novel technologies to cope with spectrum and energy scarcity: Cognitive Radio (CR) and Energy Harvesting (EH). CR is a novel paradigm for improving the spectral efficiency in wireless networks, by enabling the coexistence of an incumbent legacy system and an opportunistic system with CR capability. We investigate a technique where the CR system exploits the temporal redundancy introduced by the Hybrid Automatic Retransmission reQuest (HARQ) protocol implemented by the legacy system to perform interference cancellation, thus enhancing its own throughput. Recently, EH has been proposed to cope with energy scarcity in Wireless Sensor Networks (WSNs). Devices with EH capability harvest energy from the environment, e.g., solar, wind, heat or piezo-electric, to power their circuitry and to perform data sensing, processing and communication tasks. Due to the random energy supply, how to best manage the available energy is an open research issue. In the second part of this thesis, we design control policies for EH devices, and investigate the impact of factors such as the finite battery storage, time-correlation in the EH process and battery degradation phenomena on the performance of such systems. We cast both paradigms in a stochastic optimization framework, and investigate techniques to cope with spectrum and energy scarcity by opportunistically leveraging interference and ambient energy, respectively, whose benefits are demonstrated both by theoretical analysis and numerically. As an additional topic, we investigate the issue of channel estimation in UltraWide-Band (UWB) systems. Due to the large transmission bandwidth, the channel has been typically modeled as sparse. However, some propagation phenomena, e.g., scattering from rough surfaces and frequency distortion, are better modeled by a diffuse channel. We propose a novel Hybrid Sparse/Diffuse (HSD) channel model which captures both components, and design channel estimators based on it

Towards green computing for Internet of Things: energy oriented path and message scheduling approach

Recently, energy efficiency in sensor enabled wire-less network domain has witnessed significant attention from both academia and industries. It is an enabling technological advancement towards green computing in Internet of Things (IoT) eventually supporting sensor generated big data processing for smart cities. Related literature on energy efficiency in sensor enabled wireless network environments focuses on one aspects either energy oriented path selection or energy oriented message scheduling. The definition of path also varies in literature without considering links towards energy efficiency. In this context, this paper proposes an energy oriented path selection and message scheduling framework for sensor enabled wireless network environments. The technical novelty focuses on effective cooperation between path selection and message scheduling considering links on path, location of message sender, and number of processor in sensor towards energy efficiency. Specifically, a path selection strategy is developed based on shortest path and less number of links on path (SPLL). The location of message sender, and number of processor in specific sensor are utilized for developing a longer hops (LH) message scheduling approach. A system model is presented based on M/M/1 queuing analysis to showcase the effective cooperation of SPLL and LH towards energy efficiency. Simulation oriented comparative performance evaluation attest the energy efficiency of the proposed framework as compared to the state-of-the-art techniques considering number of energy oriented metrics

Energy-efficient mobile sink routing scheme for clustered corona-based wireless sensor networks

Wireless Sensor Networks (WSNs) are generally composed of several tiny, inexpensive and self-configured sensor nodes, which are able to communicate with each other via wireless communication devices. The main duty of the nodes is to sense data and transmit to a sink via multi- or single-hop data transmission manners. Since the sensor nodes generally are limited in power resources, they deplete their energy rapidly. In addition, sensor nodes are usually distributed in places, where may be too harsh to be accessible for human. Consequently, exchanging or recharging the power supplies of the sensor nodes is difficult. Therefore, energy efficiency is the most critical issue in design of WSN, which affects the lifetime and performance of the network. Several cluster-based schemes are proposed to enhance the energy efficiency; however, most of them generate sub-optimal clusters without considering both coverage and energy issues simultaneously. Furthermore, several mobility-based schemes are proposed in order to achieve balanced energy consumption through optimizing the sojourn time and sojourn location of Mobile Sinks (MS). Nevertheless, most of them adjust the sojourn time of MS under predictable mobility pattern. Moreover, in most of existing mobility based schemes, time limitation is not considered for optimizing the sojourn location of MS. The aim behind this research is to develop an Energy-efficient Mobile Sink Routing (EMSR) Scheme, which improves the energy efficiency. The EMSR is the incorporation of three schemes: Energyefficient based Unequal-sized Clustering (EUC) mechanism aims to construct the optimal sized clusters, which ensures the energy conservation and coverage preservation. Collaborative Mobile Sink-based Inter-Cluster Routing (CMSICR) mechanism aims to optimize the sojourn time of MS to balance the energy consumption among Cluster Heads (CH). An Energy-efficient Intra-cluster Movement of Mobile Sink (EIM2S) mechanism, which identifies the optimal sojourn locations of the MS within clusters in order to balance the energy consumption among Member Nodes (MN). The EMSR partitions the network field into optimal clusters and employs MSs in order to balance the energy consumption among CHs and MNs. Simulation results show that EMSR achieved improved performance in terms of network lifetime by 51%, total energy consumption by 28% wasted energy by 36% compared to existing schemes. In conclusion, the proposed routing scheme proves to be a viable solution for multi hop cluster based WSN