538 research outputs found
An Optimal Transmission Strategy for Kalman Filtering over Packet Dropping Links with Imperfect Acknowledgements
This paper presents a novel design methodology for optimal transmission
policies at a smart sensor to remotely estimate the state of a stable linear
stochastic dynamical system. The sensor makes measurements of the process and
forms estimates of the state using a local Kalman filter. The sensor transmits
quantized information over a packet dropping link to the remote receiver. The
receiver sends packet receipt acknowledgments back to the sensor via an
erroneous feedback communication channel which is itself packet dropping. The
key novelty of this formulation is that the smart sensor decides, at each
discrete time instant, whether to transmit a quantized version of either its
local state estimate or its local innovation. The objective is to design
optimal transmission policies in order to minimize a long term average cost
function as a convex combination of the receiver's expected estimation error
covariance and the energy needed to transmit the packets. The optimal
transmission policy is obtained by the use of dynamic programming techniques.
Using the concept of submodularity, the optimality of a threshold policy in the
case of scalar systems with perfect packet receipt acknowledgments is proved.
Suboptimal solutions and their structural results are also discussed. Numerical
results are presented illustrating the performance of the optimal and
suboptimal transmission policies.Comment: Conditionally accepted in IEEE Transactions on Control of Network
System
Optimal Energy Allocation for Kalman Filtering over Packet Dropping Links with Imperfect Acknowledgments and Energy Harvesting Constraints
This paper presents a design methodology for optimal transmission energy
allocation at a sensor equipped with energy harvesting technology for remote
state estimation of linear stochastic dynamical systems. In this framework, the
sensor measurements as noisy versions of the system states are sent to the
receiver over a packet dropping communication channel. The packet dropout
probabilities of the channel depend on both the sensor's transmission energies
and time varying wireless fading channel gains. The sensor has access to an
energy harvesting source which is an everlasting but unreliable energy source
compared to conventional batteries with fixed energy storages. The receiver
performs optimal state estimation with random packet dropouts to minimize the
estimation error covariances based on received measurements. The receiver also
sends packet receipt acknowledgments to the sensor via an erroneous feedback
communication channel which is itself packet dropping.
The objective is to design optimal transmission energy allocation at the
energy harvesting sensor to minimize either a finite-time horizon sum or a long
term average (infinite-time horizon) of the trace of the expected estimation
error covariance of the receiver's Kalman filter. These problems are formulated
as Markov decision processes with imperfect state information. The optimal
transmission energy allocation policies are obtained by the use of dynamic
programming techniques. Using the concept of submodularity, the structure of
the optimal transmission energy policies are studied. Suboptimal solutions are
also discussed which are far less computationally intensive than optimal
solutions. Numerical simulation results are presented illustrating the
performance of the energy allocation algorithms.Comment: Submitted to IEEE Transactions on Automatic Control. arXiv admin
note: text overlap with arXiv:1402.663
Deep Reinforcement Learning for Wireless Sensor Scheduling in Cyber-Physical Systems
In many Cyber-Physical Systems, we encounter the problem of remote state
estimation of geographically distributed and remote physical processes. This
paper studies the scheduling of sensor transmissions to estimate the states of
multiple remote, dynamic processes. Information from the different sensors have
to be transmitted to a central gateway over a wireless network for monitoring
purposes, where typically fewer wireless channels are available than there are
processes to be monitored. For effective estimation at the gateway, the sensors
need to be scheduled appropriately, i.e., at each time instant one needs to
decide which sensors have network access and which ones do not. To address this
scheduling problem, we formulate an associated Markov decision process (MDP).
This MDP is then solved using a Deep Q-Network, a recent deep reinforcement
learning algorithm that is at once scalable and model-free. We compare our
scheduling algorithm to popular scheduling algorithms such as round-robin and
reduced-waiting-time, among others. Our algorithm is shown to significantly
outperform these algorithms for many example scenarios
Markov Decision Processes with Applications in Wireless Sensor Networks: A Survey
Wireless sensor networks (WSNs) consist of autonomous and resource-limited
devices. The devices cooperate to monitor one or more physical phenomena within
an area of interest. WSNs operate as stochastic systems because of randomness
in the monitored environments. For long service time and low maintenance cost,
WSNs require adaptive and robust methods to address data exchange, topology
formulation, resource and power optimization, sensing coverage and object
detection, and security challenges. In these problems, sensor nodes are to make
optimized decisions from a set of accessible strategies to achieve design
goals. This survey reviews numerous applications of the Markov decision process
(MDP) framework, a powerful decision-making tool to develop adaptive algorithms
and protocols for WSNs. Furthermore, various solution methods are discussed and
compared to serve as a guide for using MDPs in WSNs
- …