Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Efficient energy management for the internet of things in smart cities

The drastic increase in urbanization over the past few years requires sustainable, efficient, and smart solutions for transportation, governance, environment, quality of life, and so on. The Internet of Things offers many sophisticated and ubiquitous applications for smart cities. The energy demand of IoT applications is increased, while IoT devices continue to grow in both numbers and requirements. Therefore, smart city solutions must have the ability to efficiently utilize energy and handle the associated challenges. Energy management is considered as a key paradigm for the realization of complex energy systems in smart cities. In this article, we present a brief overview of energy management and challenges in smart cities. We then provide a unifying framework for energy-efficient optimization and scheduling of IoT-based smart cities. We also discuss the energy harvesting in smart cities, which is a promising solution for extending the lifetime of low-power devices and its related challenges. We detail two case studies. The first one targets energy-efficient scheduling in smart homes, and the second covers wireless power transfer for IoT devices in smart cities. Simulation results for the case studies demonstrate the tremendous impact of energy-efficient scheduling optimization and wireless power transfer on the performance of IoT in smart cities

A Learning Theoretic Approach to Energy Harvesting Communication System Optimization

A point-to-point wireless communication system in which the transmitter is equipped with an energy harvesting device and a rechargeable battery, is studied. Both the energy and the data arrivals at the transmitter are modeled as Markov processes. Delay-limited communication is considered assuming that the underlying channel is block fading with memory, and the instantaneous channel state information is available at both the transmitter and the receiver. The expected total transmitted data during the transmitter's activation time is maximized under three different sets of assumptions regarding the information available at the transmitter about the underlying stochastic processes. A learning theoretic approach is introduced, which does not assume any a priori information on the Markov processes governing the communication system. In addition, online and offline optimization problems are studied for the same setting. Full statistical knowledge and causal information on the realizations of the underlying stochastic processes are assumed in the online optimization problem, while the offline optimization problem assumes non-causal knowledge of the realizations in advance. Comparing the optimal solutions in all three frameworks, the performance loss due to the lack of the transmitter's information regarding the behaviors of the underlying Markov processes is quantified

Energy efficient tdma sleep scheduling in wireless sensor networks

Abstract—Sleep scheduling is a widely used mechanism in wireless sensor networks (WSNs) to reduce the energy consumption since it can save the energy wastage caused by the idle listening state. In a traditional sleep scheduling, however, sensors have to start up numerous times in a period, and thus consume extra energy due to the state transitions. The objective of this paper is to design an energy efficient sleep scheduling for low data-rate WSNs, where sensors not only consume different amounts of energy in different states (transmit, receive, idle and sleep), but also consume energy for state transitions. We use TDMA as the MAC layer protocol, because it has the advantages of avoiding collisions, idle listening and overhearing. We first propose a novel interference-free TDMA sleep scheduling problem called contiguous link scheduling, which assigns sensors with consecutive time slots to reduce the frequency of state transitions. To tackle this problem, we then present efficient centralized and distributed algorithms that use time slots at most a constant factor of the optimum. The simulation studies corroborate the theoretical results, and show the efficiency of our proposed algorithms

Review of Optimization Problems in Wireless Sensor Networks

International audienc

Joint Routing and STDMA-based Scheduling to Minimize Delays in Grid Wireless Sensor Networks

In this report, we study the issue of delay optimization and energy efficiency in grid wireless sensor networks (WSNs). We focus on STDMA (Spatial Reuse TDMA)) scheduling, where a predefined cycle is repeated, and where each node has fixed transmission opportunities during specific slots (defined by colors). We assume a STDMA algorithm that takes advantage of the regularity of grid topology to also provide a spatially periodic coloring ("tiling" of the same color pattern). In this setting, the key challenges are: 1) minimizing the average routing delay by ordering the slots in the cycle 2) being energy efficient. Our work follows two directions: first, the baseline performance is evaluated when nothing specific is done and the colors are randomly ordered in the STDMA cycle. Then, we propose a solution, ORCHID that deliberately constructs an efficient STDMA schedule. It proceeds in two steps. In the first step, ORCHID starts form a colored grid and builds a hierarchical routing based on these colors. In the second step, ORCHID builds a color ordering, by considering jointly both routing and scheduling so as to ensure that any node will reach a sink in a single STDMA cycle. We study the performance of these solutions by means of simulations and modeling. Results show the excellent performance of ORCHID in terms of delays and energy compared to a shortest path routing that uses the delay as a heuristic. We also present the adaptation of ORCHID to general networks under the SINR interference model

A Comprehensive Analysis of Literature Reported Mac and Phy Enhancements of Zigbee and its Alliances

Wireless communication is one of the most required technologies by the common man. The strength of this technology is rigorously progressing towards several novel directions in establishing personal wireless networks mounted over on low power consuming systems. The cutting-edge communication technologies like bluetooth, WIFI and ZigBee significantly play a prime role to cater the basic needs of any individual. ZigBee is one such evolutionary technology steadily getting its popularity in establishing personal wireless networks which is built on small and low-power digital radios. Zigbee defines the physical and MAC layers built on IEEE standard. This paper presents a comprehensive survey of literature reported MAC and PHY enhancements of ZigBee and its contemporary technologies with respect to performance, power consumption, scheduling, resource management and timing and address binding. The work also discusses on the areas of ZigBee MAC and PHY towards their design for specific applications

Machine Learning in Wireless Sensor Networks: Algorithms, Strategies, and Applications

Wireless sensor networks monitor dynamic environments that change rapidly over time. This dynamic behavior is either caused by external factors or initiated by the system designers themselves. To adapt to such conditions, sensor networks often adopt machine learning techniques to eliminate the need for unnecessary redesign. Machine learning also inspires many practical solutions that maximize resource utilization and prolong the lifespan of the network. In this paper, we present an extensive literature review over the period 2002-2013 of machine learning methods that were used to address common issues in wireless sensor networks (WSNs). The advantages and disadvantages of each proposed algorithm are evaluated against the corresponding problem. We also provide a comparative guide to aid WSN designers in developing suitable machine learning solutions for their specific application challenges.Comment: Accepted for publication in IEEE Communications Surveys and Tutorial