4,674 research outputs found
Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey
Wireless sensor networks (WSNs) consist of autonomous and resource-limited
devices. The devices cooperate to monitor one or more physical phenomena within
an area of interest. WSNs operate as stochastic systems because of randomness
in the monitored environments. For long service time and low maintenance cost,
WSNs require adaptive and robust methods to address data exchange, topology
formulation, resource and power optimization, sensing coverage and object
detection, and security challenges. In these problems, sensor nodes are to make
optimized decisions from a set of accessible strategies to achieve design
goals. This survey reviews numerous applications of the Markov decision process
(MDP) framework, a powerful decision-making tool to develop adaptive algorithms
and protocols for WSNs. Furthermore, various solution methods are discussed and
compared to serve as a guide for using MDPs in WSNs
Sensor Management for Tracking in Sensor Networks
We study the problem of tracking an object moving through a network of
wireless sensors. In order to conserve energy, the sensors may be put into a
sleep mode with a timer that determines their sleep duration. It is assumed
that an asleep sensor cannot be communicated with or woken up, and hence the
sleep duration needs to be determined at the time the sensor goes to sleep
based on all the information available to the sensor. Having sleeping sensors
in the network could result in degraded tracking performance, therefore, there
is a tradeoff between energy usage and tracking performance. We design sleeping
policies that attempt to optimize this tradeoff and characterize their
performance. As an extension to our previous work in this area [1], we consider
generalized models for object movement, object sensing, and tracking cost. For
discrete state spaces and continuous Gaussian observations, we derive a lower
bound on the optimal energy-tracking tradeoff. It is shown that in the low
tracking error regime, the generated policies approach the derived lower bound
Sensor Scheduling for Energy-Efficient Target Tracking in Sensor Networks
In this paper we study the problem of tracking an object moving randomly
through a network of wireless sensors. Our objective is to devise strategies
for scheduling the sensors to optimize the tradeoff between tracking
performance and energy consumption. We cast the scheduling problem as a
Partially Observable Markov Decision Process (POMDP), where the control actions
correspond to the set of sensors to activate at each time step. Using a
bottom-up approach, we consider different sensing, motion and cost models with
increasing levels of difficulty. At the first level, the sensing regions of the
different sensors do not overlap and the target is only observed within the
sensing range of an active sensor. Then, we consider sensors with overlapping
sensing range such that the tracking error, and hence the actions of the
different sensors, are tightly coupled. Finally, we consider scenarios wherein
the target locations and sensors' observations assume values on continuous
spaces. Exact solutions are generally intractable even for the simplest models
due to the dimensionality of the information and action spaces. Hence, we
devise approximate solution techniques, and in some cases derive lower bounds
on the optimal tradeoff curves. The generated scheduling policies, albeit
suboptimal, often provide close-to-optimal energy-tracking tradeoffs
Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks
In this paper, we consider an intrusion detection application for Wireless
Sensor Networks (WSNs). We study the problem of scheduling the sleep times of
the individual sensors to maximize the network lifetime while keeping the
tracking error to a minimum. We formulate this problem as a
partially-observable Markov decision process (POMDP) with continuous
state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]).
However, unlike their formulation, we consider infinite horizon discounted and
average cost objectives as performance criteria. For each criterion, we propose
a convergent on-policy Q-learning algorithm that operates on two timescales,
while employing function approximation to handle the curse of dimensionality
associated with the underlying POMDP. Our proposed algorithm incorporates a
policy gradient update using a one-simulation simultaneous perturbation
stochastic approximation (SPSA) estimate on the faster timescale, while the
Q-value parameter (arising from a linear function approximation for the
Q-values) is updated in an on-policy temporal difference (TD) algorithm-like
fashion on the slower timescale. The feature selection scheme employed in each
of our algorithms manages the energy and tracking components in a manner that
assists the search for the optimal sleep-scheduling policy. For the sake of
comparison, in both discounted and average settings, we also develop a function
approximation analogue of the Q-learning algorithm. This algorithm, unlike the
two-timescale variant, does not possess theoretical convergence guarantees.
Finally, we also adapt our algorithms to include a stochastic iterative
estimation scheme for the intruder's mobility model. Our simulation results on
a 2-dimensional network setting suggest that our algorithms result in better
tracking accuracy at the cost of only a few additional sensors, in comparison
to a recent prior work
Coverage Protocols for Wireless Sensor Networks: Review and Future Directions
The coverage problem in wireless sensor networks (WSNs) can be generally
defined as a measure of how effectively a network field is monitored by its
sensor nodes. This problem has attracted a lot of interest over the years and
as a result, many coverage protocols were proposed. In this survey, we first
propose a taxonomy for classifying coverage protocols in WSNs. Then, we
classify the coverage protocols into three categories (i.e. coverage aware
deployment protocols, sleep scheduling protocols for flat networks, and
cluster-based sleep scheduling protocols) based on the network stage where the
coverage is optimized. For each category, relevant protocols are thoroughly
reviewed and classified based on the adopted coverage techniques. Finally, we
discuss open issues (and recommend future directions to resolve them)
associated with the design of realistic coverage protocols. Issues such as
realistic sensing models, realistic energy consumption models, realistic
connectivity models and sensor localization are covered
- …