Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algorithms

The first aim of this paper is to establish the weak convergence rate of nonlinear two-time-scale stochastic approximation algorithms. Its second aim is to introduce the averaging principle in the context of two-time-scale stochastic approximation algorithms. We first define the notion of asymptotic efficiency in this framework, then introduce the averaged two-time-scale stochastic approximation algorithm, and finally establish its weak convergence rate. We show, in particular, that both components of the averaged two-time-scale stochastic approximation algorithm simultaneously converge at the optimal rate $\sqrt{n}$.Comment: Published at http://dx.doi.org/10.1214/105051606000000448 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Optimal structured feedback policies for ABR flow control using two-timescale SPSA

Optimal structured feedback control policies for rate-based flow control of available bit rate (ABR) service in asynchronous transfer mode (ATM) networks are obtained in the presence of information and propagation delays, using a numerically efficient twotimescale simultaneous perturbation stochastic approximation (SPSA) algorithm. Models comprising both a single bottleneck node and a network with multiple bottleneck nodes are considered. A convergence analysis of the algorithm is presented. Numerical experiments demonstrate fast convergence even in the presence of significant delays. We also illustrate performance comparisons with the well-known ERICA algorithm and describe another algorithm (based on ERICA) that does not require estimating available bandwidth (as in ERICA). Key Words Optimal structured feedback policies, rate-based ABR flow control, single bottleneck node, network of nodes, two-timescale SPSA