Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Resource Allocation in Wireless Networks with RF Energy Harvesting and Transfer

Radio frequency (RF) energy harvesting and transfer techniques have recently become alternative methods to power the next generation of wireless networks. As this emerging technology enables proactive replenishment of wireless devices, it is advantageous in supporting applications with quality-of-service (QoS) requirement. This article focuses on the resource allocation issues in wireless networks with RF energy harvesting capability, referred to as RF energy harvesting networks (RF-EHNs). First, we present an overview of the RF-EHNs, followed by a review of a variety of issues regarding resource allocation. Then, we present a case study of designing in the receiver operation policy, which is of paramount importance in the RF-EHNs. We focus on QoS support and service differentiation, which have not been addressed by previous literatures. Furthermore, we outline some open research directions.Comment: To appear in IEEE Networ

Optimal Cooperative Power Allocation for Energy Harvesting Enabled Relay Networks

In this paper, we present a new power allocation scheme for a decode-and-forward (DF) relaying-enhanced cooperative wireless system. While both source and relay nodes may have limited traditional brown power supply or fixed green energy storage, the hybrid source node can also draw power from the surrounding radio frequency (RF) signals. In particular, we assume a deterministic RF energy harvesting (EH) model under which the signals transmitted by the relay serve as the renewable energy source for the source node. The amount of harvested energy is known for a given transmission power of the forwarding signal and channel condition between the source and relay nodes. To maximize the overall throughput while meeting the constraints imposed by the non-sustainable energy sources and the renewable energy source, an optimization problem is formulated and solved. Based on different harvesting efficiency and channel condition, closed form solutions are derived to obtain the optimal source and relay power allocation jointly. It is shown that instead of demanding high on-grid power supply or high green energy availability, the system can achieve compatible or higher throughput by utilizing the harvested energy

Efficient energy management for the internet of things in smart cities

The drastic increase in urbanization over the past few years requires sustainable, efficient, and smart solutions for transportation, governance, environment, quality of life, and so on. The Internet of Things offers many sophisticated and ubiquitous applications for smart cities. The energy demand of IoT applications is increased, while IoT devices continue to grow in both numbers and requirements. Therefore, smart city solutions must have the ability to efficiently utilize energy and handle the associated challenges. Energy management is considered as a key paradigm for the realization of complex energy systems in smart cities. In this article, we present a brief overview of energy management and challenges in smart cities. We then provide a unifying framework for energy-efficient optimization and scheduling of IoT-based smart cities. We also discuss the energy harvesting in smart cities, which is a promising solution for extending the lifetime of low-power devices and its related challenges. We detail two case studies. The first one targets energy-efficient scheduling in smart homes, and the second covers wireless power transfer for IoT devices in smart cities. Simulation results for the case studies demonstrate the tremendous impact of energy-efficient scheduling optimization and wireless power transfer on the performance of IoT in smart cities

Scheduling for Cooperative Energy Harvesting Sensor Networks

In cooperative communication networks, the source node transmits its data to the destination either directly or cooperatively with a cooperating node. When using energy harvesting technology, where nodes collect their energy from the environment, the energy availability at the nodes becomes unpredictable due to the stochastic nature of energy harvesting processes. As a result, when the source has a transmission, it cannot immediately transmit its data cooperatively with the cooperating node. It first needs to determine whether the cooperating node has sufficient energy to forward its transmission or not. Otherwise, its transmitted data may get lost. Therefore, when using energy harvesting, the challenge is for the source to schedule its transmissions whether directly or cooperatively, such that the fraction of its events (sensed data) that are successfully reported to the destination is maximized. Hence, in this dissertation, we address the problem of cooperating node scheduling in energy harvesting sensor networks. We consider the problem for the case of a single cooperating node and the case of multiple cooperating nodes, as well as the scenarios of one-way and two-way cooperative communications. We propose a simple scheduling scheme, called feedback scheme, which enables the source to optimally schedule its transmissions whether directly or cooperatively. We show that the feedback scheme maximizes the system performance, but does not require auxiliary parameter optimization as does the-state-of-the-art scheme, i.e., the threshold-based scheme. However, the feedback scheme has the problem of overhead caused by transmitting the energy status of the cooperating node to the source. To overcome this burden, we introduce a statistical model that enables the source to estimate the energy status of the cooperating node. Because cooperation may result in the cooperating node performing worse than the source, we address this problem through fairness in the performance between the nodes in the network. In addition, we address the problem of scheduling for throughput maximization in a wireless energy harvesting uplink. We propose centralized and distributed algorithms that find the optimal solution, and we address complexity issues. Our algorithms are shown to have a linear or quadratic complexity compared to the exponential complexity of the brute force approach. Compared with cooperative transmission, our approach maximizes the network throughput such that no node\u27s throughput is adversely affected