293 research outputs found
A Kiefer-Wolfowitz algorithm with randomized differences
©1999 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.A Kiefer-Wolfowitz or simultaneous perturbation algorithm that uses either one-sided or two-sided randomized differences and truncations at randomly varying bounds is given in this paper. At each iteration of the algorithm only two observations are required in contrast to 2l observations, where l is the dimension, in the classical algorithm, The algorithm given here is shown to he convergent under only some mild conditions. A rate of convergence and an asymptotic normality of the algorithm are also established
A companion for the Kiefer--Wolfowitz--Blum stochastic approximation algorithm
A stochastic algorithm for the recursive approximation of the location
of a maximum of a regression function was introduced by Kiefer and
Wolfowitz [Ann. Math. Statist. 23 (1952) 462--466] in the univariate framework,
and by Blum [Ann. Math. Statist. 25 (1954) 737--744] in the multivariate case.
The aim of this paper is to provide a companion algorithm to the
Kiefer--Wolfowitz--Blum algorithm, which allows one to simultaneously
recursively approximate the size of the maximum of the regression
function. A precise study of the joint weak convergence rate of both algorithms
is given; it turns out that, unlike the location of the maximum, the size of
the maximum can be approximated by an algorithm which converges at the
parametric rate. Moreover, averaging leads to an asymptotically efficient
algorithm for the approximation of the couple .Comment: Published in at http://dx.doi.org/10.1214/009053606000001451 the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
The stochastic approximation method for the estimation of a multivariate probability density
We apply the stochastic approximation method to construct a large class of
recursive kernel estimators of a probability density, including the one
introduced by Hall and Patil (1994). We study the properties of these
estimators and compare them with Rosenblatt's nonrecursive estimator. It turns
out that, for pointwise estimation, it is preferable to use the nonrecursive
Rosenblatt's kernel estimator rather than any recursive estimator. A contrario,
for estimation by confidence intervals, it is better to use a recursive
estimator rather than Rosenblatt's estimator.Comment: 28 page
Q-learning with censored data
We develop methodology for a multistage decision problem with flexible number
of stages in which the rewards are survival times that are subject to
censoring. We present a novel Q-learning algorithm that is adjusted for
censored data and allows a flexible number of stages. We provide finite sample
bounds on the generalization error of the policy learned by the algorithm, and
show that when the optimal Q-function belongs to the approximation space, the
expected survival time for policies obtained by the algorithm converges to that
of the optimal policy. We simulate a multistage clinical trial with flexible
number of stages and apply the proposed censored-Q-learning algorithm to find
individualized treatment regimens. The methodology presented in this paper has
implications in the design of personalized medicine trials in cancer and in
other life-threatening diseases.Comment: Published in at http://dx.doi.org/10.1214/12-AOS968 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
- …