4,077 research outputs found
The CMA Evolution Strategy: A Tutorial
This tutorial introduces the CMA Evolution Strategy (ES), where CMA stands
for Covariance Matrix Adaptation. The CMA-ES is a stochastic, or randomized,
method for real-parameter (continuous domain) optimization of non-linear,
non-convex functions. We try to motivate and derive the algorithm from
intuitive concepts and from requirements of non-linear, non-convex search in
continuous domain.Comment: ArXiv e-prints, arXiv:1604.xxxx
Evolutionary Algorithms for Reinforcement Learning
There are two distinct approaches to solving reinforcement learning problems,
namely, searching in value function space and searching in policy space.
Temporal difference methods and evolutionary algorithms are well-known examples
of these approaches. Kaelbling, Littman and Moore recently provided an
informative survey of temporal difference methods. This article focuses on the
application of evolutionary algorithms to the reinforcement learning problem,
emphasizing alternative policy representations, credit assignment methods, and
problem-specific genetic operators. Strengths and weaknesses of the
evolutionary approach to reinforcement learning are presented, along with a
survey of representative applications
Markov Chain Analysis of Evolution Strategies on a Linear Constraint Optimization Problem
This paper analyses a -Evolution Strategy, a randomised
comparison-based adaptive search algorithm, on a simple constraint optimisation
problem. The algorithm uses resampling to handle the constraint and optimizes a
linear function with a linear constraint. Two cases are investigated: first the
case where the step-size is constant, and second the case where the step-size
is adapted using path length control. We exhibit for each case a Markov chain
whose stability analysis would allow us to deduce the divergence of the
algorithm depending on its internal parameters. We show divergence at a
constant rate when the step-size is constant. We sketch that with step-size
adaptation geometric divergence takes place. Our results complement previous
studies where stability was assumed.Comment: Amir Hussain; Zhigang Zeng; Nian Zhang. IEEE Congress on Evolutionary
Computation, Jul 2014, Beijing, Chin
Efficient Covariance Matrix Update for Variable Metric Evolution Strategies
International audienceRandomized direct search algorithms for continuous domains, such as Evolution Strategies, are basic tools in machine learning. They are especially needed when the gradient of an objective function (e.g., loss, energy, or reward function) cannot be computed or estimated efficiently. Application areas include supervised and reinforcement learning as well as model selection. These randomized search strategies often rely on normally distributed additive variations of candidate solutions. In order to efficiently search in non-separable and ill-conditioned landscapes the covariance matrix of the normal distribution must be adapted, amounting to a variable metric method. Consequently, Covariance Matrix Adaptation (CMA) is considered state-of-the-art in Evolution Strategies. In order to sample the normal distribution, the adapted covariance matrix needs to be decomposed, requiring in general operations, where is the search space dimension. We propose a new update mechanism which can replace a rank-one covariance matrix update and the computationally expensive decomposition of the covariance matrix. The newly developed update rule reduces the computational complexity of the rank-one covariance matrix adaptation to without resorting to outdated distributions. We derive new versions of the elitist Covariance Matrix Adaptation Evolution Strategy (CMA-ES) and the multi-objective CMA-ES. These algorithms are equivalent to the original procedures except that the update step for the variable metric distribution scales better in the problem dimension. We also introduce a simplified variant of the non-elitist CMA-ES with the incremental covariance matrix update and investigate its performance. Apart from the reduced time-complexity of the distribution update, the algebraic computations involved in all new algorithms are simpler compared to the original versions. The new update rule improves the performance of the CMA-ES for large scale machine learning problems in which the objective function can be evaluated fast
The Hessian Estimation Evolution Strategy
We present a novel black box optimization algorithm called Hessian Estimation
Evolution Strategy. The algorithm updates the covariance matrix of its sampling
distribution by directly estimating the curvature of the objective function.
This algorithm design is targeted at twice continuously differentiable
problems. For this, we extend the cumulative step-size adaptation algorithm of
the CMA-ES to mirrored sampling. We demonstrate that our approach to covariance
matrix adaptation is efficient by evaluation it on the BBOB/COCO testbed. We
also show that the algorithm is surprisingly robust when its core assumption of
a twice continuously differentiable objective function is violated. The
approach yields a new evolution strategy with competitive performance, and at
the same time it also offers an interesting alternative to the usual covariance
matrix update mechanism
NEUROEVOLUTION AND AN APPLICATION OF AN AGENT BASED MODEL FOR FINANCIAL MARKET
Market prediction is one of the most difficult problems for the machine learning community. Even though, successful trading strategies can be found for the training data using various optimization methods, these strategies usually do not perform well on the test data as expected. Therefore, selection of the correct strategy becomes problematic. In this study, we propose an evolutionary algorithm that produces a variation of trader agents ensuring that the trading strategies they use are different. We discuss that because the selection of the correct strategy is difficult, a variety of agents can be used simultaneously in order to reduce risk. We simulate trader agents on real market data and attempt to optimize their actions. Agent decisions are based on Echo State Networks. The agents take various market indicators as inputs and produce an action such as: buy or sell. We optimize the parameters of the echo state networks using evolutionary algorithms
Linear Convergence of Comparison-based Step-size Adaptive Randomized Search via Stability of Markov Chains
In this paper, we consider comparison-based adaptive stochastic algorithms
for solving numerical optimisation problems. We consider a specific subclass of
algorithms that we call comparison-based step-size adaptive randomized search
(CB-SARS), where the state variables at a given iteration are a vector of the
search space and a positive parameter, the step-size, typically controlling the
overall standard deviation of the underlying search distribution.We investigate
the linear convergence of CB-SARS on\emph{scaling-invariant} objective
functions. Scaling-invariantfunctions preserve the ordering of points with
respect to their functionvalue when the points are scaled with the same
positive parameter (thescaling is done w.r.t. a fixed reference point). This
class offunctions includes norms composed with strictly increasing functions
aswell as many non quasi-convex and non-continuousfunctions. On
scaling-invariant functions, we show the existence of ahomogeneous Markov
chain, as a consequence of natural invarianceproperties of CB-SARS (essentially
scale-invariance and invariance tostrictly increasing transformation of the
objective function). We thenderive sufficient conditions for \emph{global
linear convergence} ofCB-SARS, expressed in terms of different stability
conditions of thenormalised homogeneous Markov chain (irreducibility,
positivity, Harrisrecurrence, geometric ergodicity) and thus define a general
methodologyfor proving global linear convergence of CB-SARS algorithms
onscaling-invariant functions. As a by-product we provide aconnexion between
comparison-based adaptive stochasticalgorithms and Markov chain Monte Carlo
algorithms.Comment: SIAM Journal on Optimization, Society for Industrial and Applied
Mathematics, 201
\u3cem\u3eAnolis\u3c/em\u3e Sex Chromosomes Are Derived from A Single Ancestral Pair
To explain the frequency and distribution of heteromorphic sex chromosomes in the lizard genus Anolis, we compared the relative roles of sex chromosome conservation versus turnover of sexâdetermining mechanisms. We used modelâbased comparative methods to reconstruct karyotype evolution and the presence of heteromorphic sex chromosomes onto a newly generated Anolis phylogeny. We found that heteromorphic sex chromosomes evolved multiple times in the genus. Fluorescent in situ hybridization (FISH) of repetitive DNA showed variable rates of Y chromosome degeneration among Anolis species and identified previously undetected, homomorphic sex chromosomes in two species. We confirmed homology of sex chromosomes in the genus by performing FISH of an Xâlinked bacterial artificial chromosome (BAC) and quantitative PCR of Xâlinked genes in multiple Anolis species sampled across the phylogeny. Taken together, these results are consistent with longâterm conservation of sex chromosomes in the group. Our results pave the way to address additional questions related to Anolis sex chromosome evolution and describe a conceptual framework that can be used to evaluate the origins and evolution of heteromorphic sex chromosomes in other clades
- âŠ