Evolutionary Algorithms for Reinforcement Learning

There are two distinct approaches to solving reinforcement learning problems, namely, searching in value function space and searching in policy space. Temporal difference methods and evolutionary algorithms are well-known examples of these approaches. Kaelbling, Littman and Moore recently provided an informative survey of temporal difference methods. This article focuses on the application of evolutionary algorithms to the reinforcement learning problem, emphasizing alternative policy representations, credit assignment methods, and problem-specific genetic operators. Strengths and weaknesses of the evolutionary approach to reinforcement learning are presented, along with a survey of representative applications

Dismantling Lamarckism: why descriptions of socio-economic evolution as Lamarckian are misleading

“The original publication is available at www.springerlink.com”. Copyright Springer.This paper addresses the widespread tendency to describe socio-economic evolution as Lamarckian. The difference between Lamarckian and Darwinian replication is clarified. It is shown that a phenotype-genotype distinction must first be established before we can identify Lamarckian transmission. To qualify as Lamarckian inheritance, acquired properties at the phenotypic level must be encoded in a genotype that is passed on to the next generation. Some possible social replicators (or genotypes) are identified, with a view to exploring possible distinctions between genotype and phenotype at the social level. It is concluded that the Lamarckian label does not readily transfer to socio-economic evolution, despite the fact that social genotypes (such as routines) may adapt within any given phenotype (such as an organisation). By contrast, no such problems exist with the description of socio-economic evolution as Darwinian.Peer reviewe

Impact of alife simulation of Darwinian and Lamarckian evolutionary theories

Dissertation presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Information Systems and Technologies ManagementUntil nowadays, the scientific community firmly rejected the Theory of Inheritance of Acquired Characteristics, a theory mostly associated with the name of Jean-Baptiste Lamarck (1774-1829). Though largely dismissed when applied to biological organisms, this theory found its place in a young discipline called Artificial Life. Based on the two abstract models of Darwinian and Lamarckian evolutionary theories built using neural networks and genetic algorithms, this research aims to present a notion of the potential impact of implementation of Lamarckian knowledge inheritance across disciplines. In order to obtain our results, we conducted a focus group discussion between experts in biology, computer science and philosophy, and used their opinions as qualitative data in our research. As a result of completing the above procedure, we have found some implications of such implementation in each mentioned discipline. In synthetic biology, this means that we would engineer organisms precisely up to our specific needs. At the moment, we can think of better drugs, greener fuels and dramatic changes in chemical industry. In computer science, Lamarckian evolutionary algorithms have been used for quite some years, and quite successfully. However, their application in strong ALife can only be approximated based on the existing roadmaps of futurists. In philosophy, creating artificial life seems consistent with nature and even God, if there is one. At the same time, this implementation may contradict the concept of free will, which is defined as the capacity for an agent to make choices in which the outcome has not been determined by past events. This study has certain limitations, which means that larger focus group and more prepared participants would provide more precise results

Investigations into Lamarckism, Baldwinism and Local Search in Grammatical Evolution Guided by Reinforcement

Grammatical Evolution Guided by Reinforcement is an extension of Grammatical Evolution that tries to improve the evolutionary process adding break a learning process for all the individuals in the population. With this aim, each individual is given a chance to learn through a reinforcement learning mechanism during its lifetime. The learning process is completed with a Lamarckian mechanism in which an original genotype is replaced by the best learnt genotype for the individual. In a way, Grammatical Evolution Guided by Reinforcement shares an important feature with other hybrid algorithms, i.e. global search in the evolutionary process combined with local search in the learning process. In this paper the role of the Lamarck Hypothesis is reviewed and a solution inspired only in the Baldwin effect is included as well. Besides, different techniques about the trade-off between exploitation and exploration in the reinforcement learning step followed by Grammatical Evolution Guided by Reinforcement are studied. In order to evaluate the results, the system is applied on two different domains: a simple autonomous navigation problem in a simulated Kephera robot and a typical Boolean function problem

Baldwinian accounts of language evolution

Since Hinton & Nowlan published their seminal paper (Hinton & Nowlan 1987), the neglected evolutionary process of the Baldwin effect has been widely acknowledged. Especially in the field of language evolution, the Baldwin effect (Baldwin 1896d, Simpson 1953) has been expected to salvage the long-lasting deadlocked situation of modern linguistics: i.e., it may shed light on the relationship between environment and innateness in the formation of language.However, as intense research of this evolutionary theory goes on, certain robust difficulties have become apparent. One example is genotype-phenotype correlation. By computer simulations, both Yamauchi (1999, 2001) and Mayley (19966) show that for the Baldwin effect to work legitimately, correlation between genotypes and phenotypes is the most essential underpinning. This is due to the fact that this type of the Baldwin effect adopts as its core mechanism Waddington's (1975) "genetic assimilation". In this mechanism, phenocopies have to be genetically closer to the innately predisposed genotype. Unfortunately this is an overly naiive assumption for the theory of language evolution. As a highly complex cognitive ability, the possibility that this type of genotype-phenotype correlation exists in the domain of linguistic ability is vanishingly small.In this thesis, we develop a new type of mechanism, called "Baldwinian Niche Construction (BNC), that has a rich explanatory power and can potentially over¬ come this bewildering problem of the Baldwin effect. BNC is based on the theory of niche construction that has been developed by Odling-Smee et al. (2003). The incorporation of the theory into the Baldwin effect was first suggested by Deacon (1997) and briefly introduced by Godfrey-Smith (2003). However, its formulation is yet incomplete.In the thesis, first, we review the studies of the Baldwin effect in both biology and the study of language evolution. Then the theory of BNC is more rigorously developed. Linguistic communication has an intrinsic property that is fundamentally described in the theory of niche construction. This naturally leads us to the theoretical necessity of BNC in language evolution. By creating a new linguistic niche, learning discloses a previously hidden genetic variance on which the Baldwin 'canalizing' effect can take place. It requires no genetic modification in a given genepool. There is even no need that genes responsible for learning occupy the same loci as genes for the innate linguistic knowledge. These and other aspects of BNC are presented with some results from computer simulations