Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey

Wireless sensor networks (WSNs) consist of autonomous and resource-limited devices. The devices cooperate to monitor one or more physical phenomena within an area of interest. WSNs operate as stochastic systems because of randomness in the monitored environments. For long service time and low maintenance cost, WSNs require adaptive and robust methods to address data exchange, topology formulation, resource and power optimization, sensing coverage and object detection, and security challenges. In these problems, sensor nodes are to make optimized decisions from a set of accessible strategies to achieve design goals. This survey reviews numerous applications of the Markov decision process (MDP) framework, a powerful decision-making tool to develop adaptive algorithms and protocols for WSNs. Furthermore, various solution methods are discussed and compared to serve as a guide for using MDPs in WSNs

Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks

In this paper, we consider an intrusion detection application for Wireless Sensor Networks (WSNs). We study the problem of scheduling the sleep times of the individual sensors to maximize the network lifetime while keeping the tracking error to a minimum. We formulate this problem as a partially-observable Markov decision process (POMDP) with continuous state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]). However, unlike their formulation, we consider infinite horizon discounted and average cost objectives as performance criteria. For each criterion, we propose a convergent on-policy Q-learning algorithm that operates on two timescales, while employing function approximation to handle the curse of dimensionality associated with the underlying POMDP. Our proposed algorithm incorporates a policy gradient update using a one-simulation simultaneous perturbation stochastic approximation (SPSA) estimate on the faster timescale, while the Q-value parameter (arising from a linear function approximation for the Q-values) is updated in an on-policy temporal difference (TD) algorithm-like fashion on the slower timescale. The feature selection scheme employed in each of our algorithms manages the energy and tracking components in a manner that assists the search for the optimal sleep-scheduling policy. For the sake of comparison, in both discounted and average settings, we also develop a function approximation analogue of the Q-learning algorithm. This algorithm, unlike the two-timescale variant, does not possess theoretical convergence guarantees. Finally, we also adapt our algorithms to include a stochastic iterative estimation scheme for the intruder's mobility model. Our simulation results on a 2-dimensional network setting suggest that our algorithms result in better tracking accuracy at the cost of only a few additional sensors, in comparison to a recent prior work

Smart Sleeping Policies for Energy Efficient Tracking in Sensor NetworksÂ

No Abstrac

Coverage Protocols for Wireless Sensor Networks: Review and Future Directions

The coverage problem in wireless sensor networks (WSNs) can be generally defined as a measure of how effectively a network field is monitored by its sensor nodes. This problem has attracted a lot of interest over the years and as a result, many coverage protocols were proposed. In this survey, we first propose a taxonomy for classifying coverage protocols in WSNs. Then, we classify the coverage protocols into three categories (i.e. coverage aware deployment protocols, sleep scheduling protocols for flat networks, and cluster-based sleep scheduling protocols) based on the network stage where the coverage is optimized. For each category, relevant protocols are thoroughly reviewed and classified based on the adopted coverage techniques. Finally, we discuss open issues (and recommend future directions to resolve them) associated with the design of realistic coverage protocols. Issues such as realistic sensing models, realistic energy consumption models, realistic connectivity models and sensor localization are covered

Energy-aware Scheduling of Surveillance in Wireless Multimedia Sensor Networks

Wireless sensor networks involve a large number of sensor nodes with limited energy supply, which impacts the behavior of their application. In wireless multimedia sensor networks, sensor nodes are equipped with audio and visual information collection modules. Multimedia contents are ubiquitously retrieved in surveillance applications. To solve the energy problems during target surveillance with wireless multimedia sensor networks, an energy-aware sensor scheduling method is proposed in this paper. Sensor nodes which acquire acoustic signals are deployed randomly in the sensing fields. Target localization is based on the signal energy feature provided by multiple sensor nodes, employing particle swarm optimization (PSO). During the target surveillance procedure, sensor nodes are adaptively grouped in a totally distributed manner. Specially, the target motion information is extracted by a forecasting algorithm, which is based on the hidden Markov model (HMM). The forecasting results are utilized to awaken sensor node in the vicinity of future target position. According to the two properties, signal energy feature and residual energy, the sensor nodes decide whether to participate in target detection separately with a fuzzy control approach. Meanwhile, the local routing scheme of data transmission towards the observer is discussed. Experimental results demonstrate the efficiency of energy-aware scheduling of surveillance in wireless multimedia sensor network, where significant energy saving is achieved by the sensor awakening approach and data transmission paths are calculated with low computational complexity

Energy efficient multi-target tracking in heterogeneous wireless sensor networks

Title from PDF of title page, viewed on June 3, 2011VitaIncludes bibliographical references (p. 30-32)Thesis (M.S)--School of Computing and Engineering. University of Missouri--Kansas City, 2011Tracking multiple targets in an energy efficient way is an important challenge in wireless sensor networks (WSNs). While most of the prior work consider tracking multiple targets as execution of single target tracking algorithms multiple times and utilize only single parameters for efficient energy consumption, we identify multiple parameters that can influence the energy efficiency of sensors in the WSN. We observe that there are several impacting parameters that can affect the energy efficiency of the sensors in the WSN which are: the relative location of the sensor with respect to the target's motion, multiple targets tracked by the sensor, and the remaining energy in the sensor. These impacting parameters are used to decide the tracking state of the sensors and further, our observations reveal the implications of combining these parameters and we identify that the optimal energy consumption is governed by their usage in particular network conditions. Based on these observations we proceed to propose our Adaptive Multi-Target Tracking (AMTT) algorithm that can identify the local network conditions for individual sensors in distributed environment without any centralized co-ordination, and uses required combination of impacting parameters to achieve energy efficiency.Introduction -- Related work -- Proposed multi-target tracking system -- Simulation setup and results -- Conclusions and future wor

The Coverage Problem in Video-Based Wireless Sensor Networks: A Survey

Wireless sensor networks typically consist of a great number of tiny low-cost electronic devices with limited sensing and computing capabilities which cooperatively communicate to collect some kind of information from an area of interest. When wireless nodes of such networks are equipped with a low-power camera, visual data can be retrieved, facilitating a new set of novel applications. The nature of video-based wireless sensor networks demands new algorithms and solutions, since traditional wireless sensor networks approaches are not feasible or even efficient for that specialized communication scenario. The coverage problem is a crucial issue of wireless sensor networks, requiring specific solutions when video-based sensors are employed. In this paper, it is surveyed the state of the art of this particular issue, regarding strategies, algorithms and general computational solutions. Open research areas are also discussed, envisaging promising investigation considering coverage in video-based wireless sensor networks

Energy efficient radio tomographic imaging

pre-printIn this paper, our goal is to develop approaches to reduce the energy consumption in Radio Tomographic Imaging (RTI)-based methods for device free localization without giving up localization accuracy. Our key idea is to only measure those links that are near the current location of the moving object being tracked. We propose two approaches to find the most effective links near the tracked object. In our first approach, we only consider links that are in an ellipse around the current velocity vector of the moving object. In our second approach, we only consider links that cross through a circle with radius r from the current position of the moving object. Thus, rather than creating an attenuation image of the whole area in RTI, we only create the attenuation image for effective links in a small area close to the current location of the moving object. We also develop an adaptive algorithm for determining r. We evaluate the proposed approaches in terms of energy consumption and localization error in three different test areas. Our experimental results show that using our approach, we are able to save 50% to 80% of energy. Interestingly, we find that our radius-based approach actually increases the accuracy of localization

QoS Provision for Wireless Sensor Networks

Wireless sensor network is a fast growing area of research, receiving attention not only within the computer science and electrical engineering communities, but also in relation to network optimization, scheduling, risk and reliability analysis within industrial and system engineering. The availability of micro-sensors and low-power wireless communications will enable the deployment of densely distributed sensor/actuator networks. And an integration of such system plays critical roles in many facets of human life ranging from intelligent assistants in hospitals to manufacturing process, to rescue agents in large scale disaster response, to sensor networks tracking environment phenomena, and others. The sensor nodes will perform significant signal processing, computation, and network self-configuration to achieve scalable, secure, robust and long-lived networks. More specifically, sensor nodes will do local processing to reduce energy costs, and key exchanges to ensure robust communications. These requirements pose interesting challenges for networking research. The most important technical challenge arises from the development of an integrated system which is 1)energy efficient because the system must be long-lived and operate without manual intervention, 2)reliable for data communication and robust to attackers because information security and system robustness are important in sensitive applications, such as military. Based on the above challenges, this dissertation provides Quality of Service (QoS) implementation and evaluation for the wireless sensor networks. It includes the following 3 modules, 1) energy-efficient routing, 2) energy-efficient coverage, 3). communication security. Energy-efficient routing combines the features of minimum energy consumption routing protocols with minimum computational cost routing protocols. Energy-efficient coverage provides on-demand sensing and measurement. Information security needs a security key exchange scheme to ensure reliable and robust communication links. QoS evaluation metrics and results are presented based on the above requirements