101,347 research outputs found
Stochastic Lyapunov analysis for consensus algorithms with noisy measurements
Abstract — This paper studies the coordination and consensus of networked agents in an uncertain environment. We consider a group of agents on an undirected graph with fixed topology, but differing from most existing work, each agent has only noisy measurements of its neighbors ’ states. Traditional consensus algorithms in general cannot deal with such a scenario. For consensus seeking, we introduce stochastic approximation type algorithms with a decreasing step size. We present a stochastic Lyaponuv analysis based upon the total mean potential associated with the agents. Subsequently, the so-called direction of invariance is introduced, which combined with the decay property of the stochastic Lyapunov function leads to mean square convergence of the consensus algorithm. I
A bi-level programming approach for trip matrix estimation and traffic control problems with stochastic user equilibrium link flows
This paper deals with two mathematically similar problems in transport network analysis: trip matrix estimation and traffic signal optimisation on congested road networks. These two problems are formulated as bi-level programming problems with stochastic user equilibrium assignment as the second-level programming problem. We differentiate two types of solutions in the combined matrix estimation and stochastic user equilibrium assignment problem (or, the combined signal optimisation and stochastic user equilibrium assignment problem): one is the solution to the bi-level programming problem and the other the mutually consistent solution where the two sub-problems in the combined problem are solved simultaneously. In this paper, we shall concentrate on the bi-level programming approach although we shall also consider mutually consistent solutions so as to contrast the two types of solutions. The purpose of the paper is to present a solution algorithm for the two bi-level programming problems and to test the algorithm on several networks
Multigrid methods for two-player zero-sum stochastic games
We present a fast numerical algorithm for large scale zero-sum stochastic
games with perfect information, which combines policy iteration and algebraic
multigrid methods. This algorithm can be applied either to a true finite state
space zero-sum two player game or to the discretization of an Isaacs equation.
We present numerical tests on discretizations of Isaacs equations or
variational inequalities. We also present a full multi-level policy iteration,
similar to FMG, which allows to improve substantially the computation time for
solving some variational inequalities.Comment: 31 page
Two Timescale Convergent Q-learning for Sleep--Scheduling in Wireless Sensor Networks
In this paper, we consider an intrusion detection application for Wireless
Sensor Networks (WSNs). We study the problem of scheduling the sleep times of
the individual sensors to maximize the network lifetime while keeping the
tracking error to a minimum. We formulate this problem as a
partially-observable Markov decision process (POMDP) with continuous
state-action spaces, in a manner similar to (Fuemmeler and Veeravalli [2008]).
However, unlike their formulation, we consider infinite horizon discounted and
average cost objectives as performance criteria. For each criterion, we propose
a convergent on-policy Q-learning algorithm that operates on two timescales,
while employing function approximation to handle the curse of dimensionality
associated with the underlying POMDP. Our proposed algorithm incorporates a
policy gradient update using a one-simulation simultaneous perturbation
stochastic approximation (SPSA) estimate on the faster timescale, while the
Q-value parameter (arising from a linear function approximation for the
Q-values) is updated in an on-policy temporal difference (TD) algorithm-like
fashion on the slower timescale. The feature selection scheme employed in each
of our algorithms manages the energy and tracking components in a manner that
assists the search for the optimal sleep-scheduling policy. For the sake of
comparison, in both discounted and average settings, we also develop a function
approximation analogue of the Q-learning algorithm. This algorithm, unlike the
two-timescale variant, does not possess theoretical convergence guarantees.
Finally, we also adapt our algorithms to include a stochastic iterative
estimation scheme for the intruder's mobility model. Our simulation results on
a 2-dimensional network setting suggest that our algorithms result in better
tracking accuracy at the cost of only a few additional sensors, in comparison
to a recent prior work
IGA-based Multi-Index Stochastic Collocation for random PDEs on arbitrary domains
This paper proposes an extension of the Multi-Index Stochastic Collocation
(MISC) method for forward uncertainty quantification (UQ) problems in
computational domains of shape other than a square or cube, by exploiting
isogeometric analysis (IGA) techniques. Introducing IGA solvers to the MISC
algorithm is very natural since they are tensor-based PDE solvers, which are
precisely what is required by the MISC machinery. Moreover, the
combination-technique formulation of MISC allows the straight-forward reuse of
existing implementations of IGA solvers. We present numerical results to
showcase the effectiveness of the proposed approach.Comment: version 3, version after revisio
Fixed-Form Variational Posterior Approximation through Stochastic Linear Regression
We propose a general algorithm for approximating nonstandard Bayesian
posterior distributions. The algorithm minimizes the Kullback-Leibler
divergence of an approximating distribution to the intractable posterior
distribution. Our method can be used to approximate any posterior distribution,
provided that it is given in closed form up to the proportionality constant.
The approximation can be any distribution in the exponential family or any
mixture of such distributions, which means that it can be made arbitrarily
precise. Several examples illustrate the speed and accuracy of our
approximation method in practice
- …