7,698 research outputs found
Newton based Stochastic Optimization using q-Gaussian Smoothed Functional Algorithms
We present the first q-Gaussian smoothed functional (SF) estimator of the
Hessian and the first Newton-based stochastic optimization algorithm that
estimates both the Hessian and the gradient of the objective function using
q-Gaussian perturbations. Our algorithm requires only two system simulations
(regardless of the parameter dimension) and estimates both the gradient and the
Hessian at each update epoch using these. We also present a proof of
convergence of the proposed algorithm. In a related recent work (Ghoshdastidar
et al., 2013), we presented gradient SF algorithms based on the q-Gaussian
perturbations. Our work extends prior work on smoothed functional algorithms by
generalizing the class of perturbation distributions as most distributions
reported in the literature for which SF algorithms are known to work and turn
out to be special cases of the q-Gaussian distribution. Besides studying the
convergence properties of our algorithm analytically, we also show the results
of several numerical simulations on a model of a queuing network, that
illustrate the significance of the proposed method. In particular, we observe
that our algorithm performs better in most cases, over a wide range of
q-values, in comparison to Newton SF algorithms with the Gaussian (Bhatnagar,
2007) and Cauchy perturbations, as well as the gradient q-Gaussian SF
algorithms (Ghoshdastidar et al., 2013).Comment: This is a longer of version of the paper with the same title accepted
in Automatic
Parallel Deterministic and Stochastic Global Minimization of Functions with Very Many Minima
The optimization of three problems with high dimensionality and many local minima are investigated
under five different optimization algorithms: DIRECT, simulated annealing, Spall’s SPSA algorithm, the KNITRO
package, and QNSTOP, a new algorithm developed at Indiana University
Simultaneous Perturbation Algorithms for Batch Off-Policy Search
We propose novel policy search algorithms in the context of off-policy, batch
mode reinforcement learning (RL) with continuous state and action spaces. Given
a batch collection of trajectories, we perform off-line policy evaluation using
an algorithm similar to that by [Fonteneau et al., 2010]. Using this
Monte-Carlo like policy evaluator, we perform policy search in a class of
parameterized policies. We propose both first order policy gradient and second
order policy Newton algorithms. All our algorithms incorporate simultaneous
perturbation estimates for the gradient as well as the Hessian of the
cost-to-go vector, since the latter is unknown and only biased estimates are
available. We demonstrate their practicality on a simple 1-dimensional
continuous state space problem
An algorithm for maximum likelihood estimation using an efficient method for approximating sensitivities
An algorithm for maximum likelihood (ML) estimation is developed primarily for multivariable dynamic systems. The algorithm relies on a new optimization method referred to as a modified Newton-Raphson with estimated sensitivities (MNRES). The method determines sensitivities by using slope information from local surface approximations of each output variable in parameter space. The fitted surface allows sensitivity information to be updated at each iteration with a significant reduction in computational effort compared with integrating the analytically determined sensitivity equations or using a finite-difference method. Different surface-fitting methods are discussed and demonstrated. Aircraft estimation problems are solved by using both simulated and real-flight data to compare MNRES with commonly used methods; in these solutions MNRES is found to be equally accurate and substantially faster. MNRES eliminates the need to derive sensitivity equations, thus producing a more generally applicable algorithm
- …