51 research outputs found
Incremental Stochastic Subgradient Algorithms for Convex Optimization
In this paper we study the effect of stochastic errors on two constrained
incremental sub-gradient algorithms. We view the incremental sub-gradient
algorithms as decentralized network optimization algorithms as applied to
minimize a sum of functions, when each component function is known only to a
particular agent of a distributed network. We first study the standard cyclic
incremental sub-gradient algorithm in which the agents form a ring structure
and pass the iterate in a cycle. We consider the method with stochastic errors
in the sub-gradient evaluations and provide sufficient conditions on the
moments of the stochastic errors that guarantee almost sure convergence when a
diminishing step-size is used. We also obtain almost sure bounds on the
algorithm's performance when a constant step-size is used. We then consider
\ram{the} Markov randomized incremental subgradient method, which is a
non-cyclic version of the incremental algorithm where the sequence of computing
agents is modeled as a time non-homogeneous Markov chain. Such a model is
appropriate for mobile networks, as the network topology changes across time in
these networks. We establish the convergence results and error bounds for the
Markov randomized method in the presence of stochastic errors for diminishing
and constant step-sizes, respectively
A distributed adaptive steplength stochastic approximation method for monotone stochastic Nash Games
We consider a distributed stochastic approximation (SA) scheme for computing
an equilibrium of a stochastic Nash game. Standard SA schemes employ
diminishing steplength sequences that are square summable but not summable.
Such requirements provide a little or no guidance for how to leverage
Lipschitzian and monotonicity properties of the problem and naive choices
generally do not preform uniformly well on a breadth of problems. While a
centralized adaptive stepsize SA scheme is proposed in [1] for the optimization
framework, such a scheme provides no freedom for the agents in choosing their
own stepsizes. Thus, a direct application of centralized stepsize schemes is
impractical in solving Nash games. Furthermore, extensions to game-theoretic
regimes where players may independently choose steplength sequences are limited
to recent work by Koshal et al. [2]. Motivated by these shortcomings, we
present a distributed algorithm in which each player updates his steplength
based on the previous steplength and some problem parameters. The steplength
rules are derived from minimizing an upper bound of the errors associated with
players' decisions. It is shown that these rules generate sequences that
converge almost surely to an equilibrium of the stochastic Nash game.
Importantly, variants of this rule are suggested where players independently
select steplength sequences while abiding by an overall coordination
requirement. Preliminary numerical results are seen to be promising.Comment: 8 pages, Proceedings of the American Control Conference, Washington,
201
- …