Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Energy Harvesting Networked Nodes: Measurements, Algorithms, and Prototyping

Recent advances in ultra-low-power wireless communications and in energy harvesting will soon enable energetically self-sustainable wireless devices. Networks of such devices will serve as building blocks for different Internet of Things (IoT) applications, such as searching for an object on a network of objects and continuous monitoring of object configurations. Yet, numerous challenges need to be addressed for the IoT vision to be fully realized. This thesis considers several challenges related to ultra-low-power energy harvesting networked nodes: energy source characterization, algorithm design, and node design and prototyping. Additionally, the thesis contributes to engineering education, specifically to project-based learning. We summarize our contributions to light and kinetic (motion) energy characterization for energy harvesting nodes. To characterize light energy, we conducted a first-of-its kind 16 month-long indoor light energy measurements campaign. To characterize energy of motion, we collected over 200 hours of human and object motion traces. We also analyzed traces previously collected in a study with over 40 participants. We summarize our insights, including light and motion energy budgets, variability, and influencing factors. These insights are useful for designing energy harvesting nodes and energy harvesting adaptive algorithms. We shared with the community our light energy traces, which can be used as energy inputs to system and algorithm simulators and emulators. We also discuss resource allocation problems we considered for energy harvesting nodes. Inspired by the needs of tracking and monitoring IoT applications, we formulated and studied resource allocation problems aimed at allocating the nodes' time-varying resources in a uniform way with respect to time. We mainly considered deterministic energy profile and stochastic environmental energy models, and focused on single node and link scenarios. We formulated optimization problems using utility maximization and lexicographic maximization frameworks, and introduced algorithms for solving the formulated problems. For several settings, we provided low-complexity solution algorithms. We also examined many simple policies. We demonstrated, analytically and via simulations, that in many settings simple policies perform well. We also summarize our design and prototyping efforts for a new class of ultra-low-power nodes - Energy Harvesting Active Networked Tags (EnHANTs). Future EnHANTs will be wireless nodes that can be attached to commonplace objects (books, furniture, clothing). We describe the EnHANTs prototypes and the EnHANTs testbed that we developed, in collaboration with other research groups, over the last 4 years in 6 integration phases. The prototypes harvest energy of the indoor light, communicate with each other via ultra-low-power transceivers, form small multihop networks, and adapt their communications and networking to their energy harvesting states. The EnHANTs testbed can expose the prototypes to light conditions based on real-world light energy traces. Using the testbed and our light energy traces, we evaluated some of our energy harvesting adaptive policies. Our insights into node design and performance evaluations may apply beyond EnHANTs to networks of various energy harvesting nodes. Finally, we present our contributions to engineering education. Over the last 4 years, we engaged high school, undergraduate, and M.S. students in more than 100 research projects within the EnHANTs project. We summarize our approaches to facilitating student learning, and discuss the results of evaluation surveys that demonstrate the effectiveness of our approaches

Markov decision processes with applications in wireless sensor networks: A survey

Ministry of Education, Singapore under its Academic Research Funding Tier

Distributed Optimal Lexicographic Max-Min Rate Allocation in Solar-Powered Wireless Sensor Networks

Understanding the optimal usage of fluctuating renewable energy in Wireless Sensor Networks (WSNs) is complex. Lexicographic Max-min (LM) rate allocation is a good solution, but is non-trivial for multi-hop WSNs, as both fairness and sensing rates have to be optimized through the exploration of all possible forwarding routes in the network. All current optimal approaches to this problem are centralized and off-line, suffering from low scalability and large computational complexity; typically solving O(N2 ) linear programming problems for N-node WSNs. This paper presents the first optimal distributed solution to this problem with much lower complexity. We apply it to Solar Powered WSNs (SP-WSNs) to achieve both LM optimality and sustainable operation. Based on realistic models of both time-varying solar power and photovoltaic-battery hardware, we propose an optimization framework that integrates a local power management algorithm with a global distributed LM rate allocation scheme. The optimality, convergence, and efficiency of our approaches are formally proven. We also evaluate our algorithms via experiments on both solar-powered MicaZ motes and extensive simulations using real solar energy data and practical power parameter settings. The results verify our theoretical analysis and demonstrate how our approach outperforms both the state-of-the-art centralized optimal and distributed heuristic solutions

Distributed Optimal Lexicographic Max-Min Rate Allocation in Solar-Powered Wireless Sensor Networks

Understanding the optimal usage of fluctuating renewable energy in wireless sensor networks (WSNs) is complex. Lexicographic max-min (LM) rate allocation is a good solution but is nontrivial for multihop WSNs, as both fairness and sensing rates have to be optimized through the exploration of all possible forwarding routes in the network. All current optimal approaches to this problem are centralized and offline, suffering from low scalability and large computational complexity—typically solving O( N 2 ) linear programming problems for N -node WSNs. This article presents the first optimal distributed solution to this problem with much lower complexity. We apply it to solar-powered wireless sensor networks (SP-WSNs) to achieve both LM optimality and sustainable operation. Based on realistic models of both time-varying solar power and photovoltaic-battery hardware, we propose an optimization framework that integrates a local power management algorithm with a global distributed LM rate allocation scheme. The optimality, convergence, and efficiency of our approaches are formally proven. We also evaluate our algorithms via experiments on both solar-powered MICAz motes and extensive simulations using real solar energy data and practical power parameter settings. The results verify our theoretical analysis and demonstrate how our approach outperforms both the state-of-the-art centralized optimal and distributed heuristic solutions. </jats:p

Distributed Optimal Lexicographic Max-Min Rate Allocation in Solar-Powered Wireless Sensor Networks

Understanding the optimal usage of fluctuating renewable energy in Wireless Sensor Networks (WSNs) is complex. Lexicographic Max-min (LM) rate allocation is a good solution, but is non-trivial for multi-hop WSNs, as both fairness and sensing rates have to be optimized through the exploration of all possible forwarding routes in the network. All current optimal approaches to this problem are centralized and off-line, suffering from low scalability and large computational complexity; typically solving O(N2 ) linear programming problems for N-node WSNs. This paper presents the first optimal distributed solution to this problem with much lower complexity. We apply it to Solar Powered WSNs (SP-WSNs) to achieve both LM optimality and sustainable operation. Based on realistic models of both time-varying solar power and photovoltaic-battery hardware, we propose an optimization framework that integrates a local power management algorithm with a global distributed LM rate allocation scheme. The optimality, convergence, and efficiency of our approaches are formally proven. We also evaluate our algorithms via experiments on both solar-powered MicaZ motes and extensive simulations using real solar energy data and practical power parameter settings. The results verify our theoretical analysis and demonstrate how our approach outperforms both the state-of-the-art centralized optimal and distributed heuristic solutions

COOPERATIVE NETWORKING AND RELATED ISSUES: STABILITY, ENERGY HARVESTING, AND NEIGHBOR DISCOVERY

This dissertation deals with various newly emerging topics in the context of cooperative networking. The first part is about the cognitive radio. To guarantee the performance of high priority users, it is important to know the activity of the high priority communication system but the knowledge is usually imperfect due to randomness in the observed signal. In such a context, the stability property of cognitive radio systems in the presence of sensing errors is studied. General guidelines on controlling the operating point of the sensing device over its receiver operating characteristics are also given. We then consider the hybrid of different modes of operation for cognitive radio systems with time-varying connectivity. The random connectivity gives additional chances that can be utilized by the low priority communication system. The second part of this dissertation is about the random access. We are specifically interested in the scenario when the nodes are harvesting energy from the environment. For such a system, we accurately assess the effect of limited, but renewable, energy availability on the stability region. The effect of finite capacity batteries is also studied. We next consider the exploitation of diversity amongst users under random access framework. That is, each user adapts its transmission probability based on the local channel state information in a decentralized manner. The impact of imperfect channel state information on the stability region is investigated. Furthermore, it is compared to the class of stationary scheduling policies that make centralized decisions based on the channel state feedback. The backpressure policy for cross-layer control of wireless multi-hop networks is known to be throughput-optimal for i.i.d. arrivals. The third part of this dissertation is about the backpressure-based control for networks with time-correlated arrivals that may exhibit long-range dependency. It is shown that the original backpressure policy is still throughput-optimal but with increased average network delay. The case when the arrival rate vector is possibly outside the stability region is also studied by augmenting the backpressure policy with the flow control mechanism. Lastly, the problem of neighbor discovery in a wireless sensor network is dealt. We first introduce the realistic effect of physical layer considerations in the evaluation of the performance of logical discovery algorithms by incorporating physical layer parameters. Secondly, given the lack of knowledge of the number of neighbors along with the lack of knowledge of the individual signal parameters, we adopt the viewpoint of random set theory to the problem of detecting the transmitting neighbors. Random set theory is a generalization of standard probability theory by assigning sets, rather than values, to random outcomes and it has been applied to multi-user detection problem when the set of transmitters are unknown and dynamically changing

Energy efficient data collection and dissemination protocols in self-organised wireless sensor networks

Wireless sensor networks (WSNs) are used for event detection and data collection in a plethora of environmental monitoring applications. However a critical factor limits the extension of WSNs into new application areas: energy constraints. This thesis develops self-organising energy efficient data collection and dissemination protocols in order to support WSNs in event detection and data collection and thus extend the use of sensor-based networks to many new application areas. Firstly, a Dual Prediction and Probabilistic Scheduler (DPPS) is developed. DPPS uses a Dual Prediction Scheme combining compression and load balancing techniques in order to manage sensor usage more efficiently. DPPS was tested and evaluated through computer simulations and empirical experiments. Results showed that DPPS reduces energy consumption in WSNs by up to 35% while simultaneously maintaining data quality and satisfying a user specified accuracy constraint. Secondly, an Adaptive Detection-driven Ad hoc Medium Access Control (ADAMAC) protocol is developed. ADAMAC limits the Data Forwarding Interruption problem which causes increased end-to-end delay and energy consumption in multi-hop sensor networks. ADAMAC uses early warning alarms to dynamically adapt the sensing intervals and communication periods of a sensor according to the likelihood of any new events occurring. Results demonstrated that compared to previous protocols such as SMAC, ADAMAC dramatically reduces end-to-end delay while still limiting energy consumption during data collection and dissemination. The protocols developed in this thesis, DPPS and ADAMAC, effectively alleviate the energy constraints associated with WSNs and will support the extension of sensorbased networks to many more application areas than had hitherto been readily possible