Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Energy Harvesting Wireless Communications: A Review of Recent Advances

This article summarizes recent contributions in the broad area of energy harvesting wireless communications. In particular, we provide the current state of the art for wireless networks composed of energy harvesting nodes, starting from the information-theoretic performance limits to transmission scheduling policies and resource allocation, medium access and networking issues. The emerging related area of energy transfer for self-sustaining energy harvesting wireless networks is considered in detail covering both energy cooperation aspects and simultaneous energy and information transfer. Various potential models with energy harvesting nodes at different network scales are reviewed as well as models for energy consumption at the nodes.Comment: To appear in the IEEE Journal of Selected Areas in Communications (Special Issue: Wireless Communications Powered by Energy Harvesting and Wireless Energy Transfer

The Beauty of the Commons: Optimal Load Sharing by Base Station Hopping in Wireless Sensor Networks

In wireless sensor networks (WSNs), the base station (BS) is a critical sensor node whose failure causes severe data losses. Deploying multiple fixed BSs improves the robustness, yet requires all BSs to be installed with large batteries and large energy-harvesting devices due to the high energy consumption of BSs. In this paper, we propose a scheme to coordinate the multiple deployed BSs such that the energy supplies required by individual BSs can be substantially reduced. In this scheme, only one BS is selected to be active at a time and the other BSs act as regular sensor nodes. We first present the basic architecture of our system, including how we keep the network running with only one active BS and how we manage the handover of the role of the active BS. Then, we propose an algorithm for adaptively selecting the active BS under the spatial and temporal variations of energy resources. This algorithm is simple to implement but is also asymptotically optimal under mild conditions. Finally, by running simulations and real experiments on an outdoor testbed, we verify that the proposed scheme is energy-efficient, has low communication overhead and reacts rapidly to network changes

Energy harvesting-aware design of wireless networks

Recent advances in low-power electronics and energy-harvesting (EH) technologies enable the design of self-sustained devices that collect part, or all, of the needed energy from the environment. Several systems can take advantage of EH, ranging from portable devices to wireless sensor networks (WSNs). While conventional design for battery-powered systems is mainly concerned with the battery lifetime, a key advantage of EH is that it enables potential perpetual operation of the devices, without requiring maintenance for battery substitutions. However, the inherent unpredictability regarding the amount of energy that can be collected from the environment might cause temporary energy shortages, which might prevent the devices to operate regularly. This uncertainty calls for the development of energy management techniques that are tailored to the EH dynamics. While most previous work on EH-capable systems has focused on energy management for single devices, the main contributions of this dissertation is the analysis and design of medium access control (MAC) protocols for WSNs operated by EH-capable devices. In particular, the dissertation first considers random access MAC protocols for single-hop EH networks, in which a fusion center collects data from a set of nodes distributed in its surrounding. MAC protocols commonly used in WSNs, such as time division multiple access (TDMA), framed-ALOHA (FA) and dynamic-FA (DFA) are investigated in the presence of EH-capable devices. A new ALOHA-based MAC protocol tailored to EH-networks, referred to as energy group-DFA (EG-DFA), is then proposed. In EG-DFA nodes with similar energy availability are grouped together and access the channel independently from other groups. It is shown that EG-DFA significantly outperforms the DFA protocol. Centralized scheduling-based MAC protocols for single-hop EH-networks with communication resource constraints are considered next. Two main scenarios are addressed, namely: i) nodes exclusively powered via EH; ii) nodes powered by a hybrid energy storage system, which is composed by a non-rechargeable battery and a capacitor charged via EH. For the former case the goal is the maximization of the network throughput, while in the latter the aim is maximizing the lifetime of the non-rechargeable batteries. For both scenarios optimal scheduling policies are derived by assuming different levels of information available at the fusion center about the energy availability at the nodes. When optimal policies are not derived explicitly, suboptimal policies are proposed and compared with performance upper bounds. Energy management policies for single devices have been investigated as well by focusing on radio frequency identification (RFID) systems, when the latter are operated by enhanced RFID tags with energy harvesting capabilities

Towards self-powered wireless sensor networks

Ubiquitous computing aims at creating smart environments in which computational and communication capabilities permeate the word at all scales, improving the human experience and quality of life in a totally unobtrusive yet completely reliable manner. According to this vision, an huge variety of smart devices and products (e.g., wireless sensor nodes, mobile phones, cameras, sensors, home appliances and industrial machines) are interconnected to realize a network of distributed agents that continuously collect, process, share and transport information. The impact of such technologies in our everyday life is expected to be massive, as it will enable innovative applications that will profoundly change the world around us. Remotely monitoring the conditions of patients and elderly people inside hospitals and at home, preventing catastrophic failures of buildings and critical structures, realizing smart cities with sustainable management of traffic and automatic monitoring of pollution levels, early detecting earthquake and forest fires, monitoring water quality and detecting water leakages, preventing landslides and avalanches are just some examples of life-enhancing applications made possible by smart ubiquitous computing systems. To turn this vision into a reality, however, new raising challenges have to be addressed, overcoming the limits that currently prevent the pervasive deployment of smart devices that are long lasting, trusted, and fully autonomous. In particular, the most critical factor currently limiting the realization of ubiquitous computing is energy provisioning. In fact, embedded devices are typically powered by short-lived batteries that severely affect their lifespan and reliability, often requiring expensive and invasive maintenance. In this PhD thesis, we investigate the use of energy-harvesting techniques to overcome the energy bottleneck problem suffered by embedded devices, particularly focusing on Wireless Sensor Networks (WSNs), which are one of the key enablers of pervasive computing systems. Energy harvesting allows to use energy readily available from the environment (e.g., from solar light, wind, body movements, etc.) to significantly extend the typical lifetime of low-power devices, enabling ubiquitous computing systems that can last virtually forever. However, the design challenges posed both at the hardware and at the software levels by the design of energy-autonomous devices are many. This thesis addresses some of the most challenging problems of this emerging research area, such as devising mechanisms for energy prediction and management, improving the efficiency of the energy scavenging process, developing protocols for harvesting-aware resource allocation, and providing solutions that enable robust and reliable security support. %, including the design of mechanisms for energy prediction and management, improving the efficiency of the energy harvesting process, the develop of protocols for harvesting-aware resource allocation, and providing solutions that enable robust and reliable security support