73,564 research outputs found
Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks
In this paper, we consider an intrusion detection application for Wireless
Sensor Networks (WSNs). We study the problem of scheduling the sleep times of
the individual sensors to maximize the network lifetime while keeping the
tracking error to a minimum. We formulate this problem as a
partially-observable Markov decision process (POMDP) with continuous
state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]).
However, unlike their formulation, we consider infinite horizon discounted and
average cost objectives as performance criteria. For each criterion, we propose
a convergent on-policy Q-learning algorithm that operates on two timescales,
while employing function approximation to handle the curse of dimensionality
associated with the underlying POMDP. Our proposed algorithm incorporates a
policy gradient update using a one-simulation simultaneous perturbation
stochastic approximation (SPSA) estimate on the faster timescale, while the
Q-value parameter (arising from a linear function approximation for the
Q-values) is updated in an on-policy temporal difference (TD) algorithm-like
fashion on the slower timescale. The feature selection scheme employed in each
of our algorithms manages the energy and tracking components in a manner that
assists the search for the optimal sleep-scheduling policy. For the sake of
comparison, in both discounted and average settings, we also develop a function
approximation analogue of the Q-learning algorithm. This algorithm, unlike the
two-timescale variant, does not possess theoretical convergence guarantees.
Finally, we also adapt our algorithms to include a stochastic iterative
estimation scheme for the intruder's mobility model. Our simulation results on
a 2-dimensional network setting suggest that our algorithms result in better
tracking accuracy at the cost of only a few additional sensors, in comparison
to a recent prior work
Extremum Seeking-based Iterative Learning Linear MPC
In this work we study the problem of adaptive MPC for linear time-invariant
uncertain models. We assume linear models with parametric uncertainties, and
propose an iterative multi-variable extremum seeking (MES)-based learning MPC
algorithm to learn on-line the uncertain parameters and update the MPC model.
We show the effectiveness of this algorithm on a DC servo motor control
example.Comment: To appear at the IEEE MSC 201
Massive MIMO-based Localization and Mapping Exploiting Phase Information of Multipath Components
In this paper, we present a robust multipath-based localization and mapping
framework that exploits the phases of specular multipath components (MPCs)
using a massive multiple-input multiple-output (MIMO) array at the base
station. Utilizing the phase information related to the propagation distances
of the MPCs enables the possibility of localization with extraordinary accuracy
even with limited bandwidth. The specular MPC parameters along with the
parameters of the noise and the dense multipath component (DMC) are tracked
using an extended Kalman filter (EKF), which enables to preserve the
distance-related phase changes of the MPC complex amplitudes. The DMC comprises
all non-resolvable MPCs, which occur due to finite measurement aperture. The
estimation of the DMC parameters enhances the estimation quality of the
specular MPCs and therefore also the quality of localization and mapping. The
estimated MPC propagation distances are subsequently used as input to a
distance-based localization and mapping algorithm. This algorithm does not need
prior knowledge about the surrounding environment and base station position.
The performance is demonstrated with real radio-channel measurements using an
antenna array with 128 ports at the base station side and a standard cellular
signal bandwidth of 40 MHz. The results show that high accuracy localization is
possible even with such a low bandwidth.Comment: 14 pages (two columns), 13 figures. This work has been submitted to
the IEEE Transaction on Wireless Communications for possible publication.
Copyright may be transferred without notice, after which this version may no
longer be accessibl
Distributed Maximum Likelihood for Simultaneous Self-localization and Tracking in Sensor Networks
We show that the sensor self-localization problem can be cast as a static
parameter estimation problem for Hidden Markov Models and we implement fully
decentralized versions of the Recursive Maximum Likelihood and on-line
Expectation-Maximization algorithms to localize the sensor network
simultaneously with target tracking. For linear Gaussian models, our algorithms
can be implemented exactly using a distributed version of the Kalman filter and
a novel message passing algorithm. The latter allows each node to compute the
local derivatives of the likelihood or the sufficient statistics needed for
Expectation-Maximization. In the non-linear case, a solution based on local
linearization in the spirit of the Extended Kalman Filter is proposed. In
numerical examples we demonstrate that the developed algorithms are able to
learn the localization parameters.Comment: shorter version is about to appear in IEEE Transactions of Signal
Processing; 22 pages, 15 figure
- …