3,654 research outputs found
Particle algorithms for optimization on binary spaces
We discuss a unified approach to stochastic optimization of pseudo-Boolean
objective functions based on particle methods, including the cross-entropy
method and simulated annealing as special cases. We point out the need for
auxiliary sampling distributions, that is parametric families on binary spaces,
which are able to reproduce complex dependency structures, and illustrate their
usefulness in our numerical experiments. We provide numerical evidence that
particle-driven optimization algorithms based on parametric families yield
superior results on strongly multi-modal optimization problems while local
search heuristics outperform them on easier problems
Evolutionary programming with q-Gaussian mutation for dynamic optimization problems
This article is posted here with permission from IEEE - Copyright @ 2008 IEEEThe use of evolutionary programming algorithms with self-adaptation of the mutation distribution for dynamic optimization problems is investigated in this paper. In the proposed method, the q-Gaussian distribution is employed to generate new candidate solutions by mutation. A real parameter q, which defines the shape of the distribution, is encoded in the chromosome of individuals and is allowed to evolve. Algorithms with self-adapted mutation generated from isotropic and anisotropic distributions are presented. In the experimental study, the q-Gaussian mutation is compared to Gaussian and Cauchy mutation on three dynamic optimization problems.This work was supported by Brazil FAPESP under Grant 04/04289-6 and UK EPSRC under Grant No. EP/E060722/01
Rank-normalization, folding, and localization: An improved for assessing convergence of MCMC
Markov chain Monte Carlo is a key computational tool in Bayesian statistics,
but it can be challenging to monitor the convergence of an iterative stochastic
algorithm. In this paper we show that the convergence diagnostic
of Gelman and Rubin (1992) has serious flaws. Traditional will
fail to correctly diagnose convergence failures when the chain has a heavy tail
or when the variance varies across the chains. In this paper we propose an
alternative rank-based diagnostic that fixes these problems. We also introduce
a collection of quantile-based local efficiency measures, along with a
practical approach for computing Monte Carlo error estimates for quantiles. We
suggest that common trace plots should be replaced with rank plots from
multiple chains. Finally, we give recommendations for how these methods should
be used in practice.Comment: Minor revision for improved clarit
Rank-normalization, folding, and localization: An improved for assessing convergence of MCMC
Markov chain Monte Carlo is a key computational tool in Bayesian statistics,
but it can be challenging to monitor the convergence of an iterative stochastic
algorithm. In this paper we show that the convergence diagnostic
of Gelman and Rubin (1992) has serious flaws. Traditional will
fail to correctly diagnose convergence failures when the chain has a heavy tail
or when the variance varies across the chains. In this paper we propose an
alternative rank-based diagnostic that fixes these problems. We also introduce
a collection of quantile-based local efficiency measures, along with a
practical approach for computing Monte Carlo error estimates for quantiles. We
suggest that common trace plots should be replaced with rank plots from
multiple chains. Finally, we give recommendations for how these methods should
be used in practice.Comment: Minor revision for improved clarit
Towards Machine Wald
The past century has seen a steady increase in the need of estimating and
predicting complex systems and making (possibly critical) decisions with
limited information. Although computers have made possible the numerical
evaluation of sophisticated statistical models, these models are still designed
\emph{by humans} because there is currently no known recipe or algorithm for
dividing the design of a statistical model into a sequence of arithmetic
operations. Indeed enabling computers to \emph{think} as \emph{humans} have the
ability to do when faced with uncertainty is challenging in several major ways:
(1) Finding optimal statistical models remains to be formulated as a well posed
problem when information on the system of interest is incomplete and comes in
the form of a complex combination of sample data, partial knowledge of
constitutive relations and a limited description of the distribution of input
random variables. (2) The space of admissible scenarios along with the space of
relevant information, assumptions, and/or beliefs, tend to be infinite
dimensional, whereas calculus on a computer is necessarily discrete and finite.
With this purpose, this paper explores the foundations of a rigorous framework
for the scientific computation of optimal statistical estimators/models and
reviews their connections with Decision Theory, Machine Learning, Bayesian
Inference, Stochastic Optimization, Robust Optimization, Optimal Uncertainty
Quantification and Information Based Complexity.Comment: 37 page
A Riemannian low-rank method for optimization over semidefinite matrices with block-diagonal constraints
We propose a new algorithm to solve optimization problems of the form for a smooth function under the constraints that is positive
semidefinite and the diagonal blocks of are small identity matrices. Such
problems often arise as the result of relaxing a rank constraint (lifting). In
particular, many estimation tasks involving phases, rotations, orthonormal
bases or permutations fit in this framework, and so do certain relaxations of
combinatorial problems such as Max-Cut. The proposed algorithm exploits the
facts that (1) such formulations admit low-rank solutions, and (2) their
rank-restricted versions are smooth optimization problems on a Riemannian
manifold. Combining insights from both the Riemannian and the convex geometries
of the problem, we characterize when second-order critical points of the smooth
problem reveal KKT points of the semidefinite problem. We compare against state
of the art, mature software and find that, on certain interesting problem
instances, what we call the staircase method is orders of magnitude faster, is
more accurate and scales better. Code is available.Comment: 37 pages, 3 figure
- …