28,289 research outputs found
Myths and Legends of the Baldwin Effect
This position paper argues that the Baldwin effect is widely
misunderstood by the evolutionary computation community. The
misunderstandings appear to fall into two general categories.
Firstly, it is commonly believed that the Baldwin effect is
concerned with the synergy that results when there is an evolving
population of learning individuals. This is only half of the story.
The full story is more complicated and more interesting. The Baldwin
effect is concerned with the costs and benefits of lifetime
learning by individuals in an evolving population. Several
researchers have focussed exclusively on the benefits, but there
is much to be gained from attention to the costs. This paper explains
the two sides of the story and enumerates ten of the costs and
benefits of lifetime learning by individuals in an evolving population.
Secondly, there is a cluster of misunderstandings about the relationship
between the Baldwin effect and Lamarckian inheritance of acquired
characteristics. The Baldwin effect is not Lamarckian. A Lamarckian
algorithm is not better for most evolutionary computing problems than
a Baldwinian algorithm. Finally, Lamarckian inheritance is not a
better model of memetic (cultural) evolution than the Baldwin effect
How to shift bias: Lessons from the Baldwin effect
An inductive learning algorithm takes a set of data as input and generates a hypothesis as
output. A set of data is typically consistent with an infinite number of hypotheses;
therefore, there must be factors other than the data that determine the output of the
learning algorithm. In machine learning, these other factors are called the bias of the
learner. Classical learning algorithms have a fixed bias, implicit in their design. Recently
developed learning algorithms dynamically adjust their bias as they search for a
hypothesis. Algorithms that shift bias in this manner are not as well understood as
classical algorithms. In this paper, we show that the Baldwin effect has implications for
the design and analysis of bias shifting algorithms. The Baldwin effect was proposed in
1896, to explain how phenomena that might appear to require Lamarckian evolution
(inheritance of acquired characteristics) can arise from purely Darwinian evolution.
Hinton and Nowlan presented a computational model of the Baldwin effect in 1987. We
explore a variation on their model, which we constructed explicitly to illustrate the lessons
that the Baldwin effect has for research in bias shifting algorithms. The main lesson is that
it appears that a good strategy for shift of bias in a learning algorithm is to begin with a
weak bias and gradually shift to a strong bias
Differential evolution with an evolution path: a DEEP evolutionary algorithm
Utilizing cumulative correlation information already existing in an evolutionary process, this paper proposes a predictive approach to the reproduction mechanism of new individuals for differential evolution (DE) algorithms. DE uses a distributed model (DM) to generate new individuals, which is relatively explorative, whilst evolution strategy (ES) uses a centralized model (CM) to generate offspring, which through adaptation retains a convergence momentum. This paper adopts a key feature in the CM of a covariance matrix adaptation ES, the cumulatively learned evolution path (EP), to formulate a new evolutionary algorithm (EA) framework, termed DEEP, standing for DE with an EP. Without mechanistically combining two CM and DM based algorithms together, the DEEP framework offers advantages of both a DM and a CM and hence substantially enhances performance. Under this architecture, a self-adaptation mechanism can be built inherently in a DEEP algorithm, easing the task of predetermining algorithm control parameters. Two DEEP variants are developed and illustrated in the paper. Experiments on the CEC'13 test suites and two practical problems demonstrate that the DEEP algorithms offer promising results, compared with the original DEs and other relevant state-of-the-art EAs
Temporal difference learning with interpolated table value functions
This paper introduces a novel function approximation architecture especially well suited to temporal difference learning. The architecture is based on using sets of interpolated table look-up functions. These offer rapid and stable learning, and are efficient when the number of inputs is small. An empirical investigation is conducted to test their performance on a supervised learning task, and on themountain car problem, a standard reinforcement learning benchmark. In each case, the interpolated table functions offer competitive performance. ©2009 IEEE
Investigating learning rates for evolution and temporal difference learning
Evidently, any learning algorithm can only learn on the basis of the information given to it. This paper presents a first attempt to place an upper bound on the information rates attainable with standard co-evolution and with TDL. The upper bound for TDL is shown to be much higher than for coevolution. Under commonly used settings for learning to play Othello for example, TDL may have an upper bound that is hundreds or even thousands of times higher than that of coevolution. To test how well these bounds correlate with actual learning rates, a simple two-player game called Treasure Hunt. is developed. While the upper bounds cannot be used to predict the number of games required to learn the optimal policy, they do correctly predict the rank order of the number of games required by each algorithm. © 2008 IEEE
- …