Investigations into Lamarckism, Baldwinism and Local Search in Grammatical Evolution Guided by Reinforcement

Grammatical Evolution Guided by Reinforcement is an extension of Grammatical Evolution that tries to improve the evolutionary process adding break a learning process for all the individuals in the population. With this aim, each individual is given a chance to learn through a reinforcement learning mechanism during its lifetime. The learning process is completed with a Lamarckian mechanism in which an original genotype is replaced by the best learnt genotype for the individual. In a way, Grammatical Evolution Guided by Reinforcement shares an important feature with other hybrid algorithms, i.e. global search in the evolutionary process combined with local search in the learning process. In this paper the role of the Lamarck Hypothesis is reviewed and a solution inspired only in the Baldwin effect is included as well. Besides, different techniques about the trade-off between exploitation and exploration in the reinforcement learning step followed by Grammatical Evolution Guided by Reinforcement are studied. In order to evaluate the results, the system is applied on two different domains: a simple autonomous navigation problem in a simulated Kephera robot and a typical Boolean function problem

Agent-Based Model of the Spectrum Auctions with Sensing Imperfections in Dynamic Spectrum Access Networks

Cognitive radio (CR) is the underlying platform for the application of dynamic spectrum access (DSA) networks. Although the auction theory and spectrum trading mechanisms have been discussed in the CR related works, their joint techno-economic impact on the efficiency of distributed CR networks has not been researched yet. In this paper we assume heterogeneous primary channels with network availability statistics unknown to each secondary user (SU) terminal. In order to detect the idle primary user (PU) network channels, the SU terminals trigger regularly the spectrum sensing mechanism and make the cooperative decision regarding the channel status at the fusion center. The imperfections of the spectrum mechanism create the possibility of the channel collision, resulting in the existence of the risk (in terms of user collision) in the network. The spectrum trading within SU network is governed by the application of the sealed-bid first-price auction, which takes into account the channel valuation as well as the statistical probability of the risk existence. In order to maximize the long-term payoff, the SU terminals take an advantage of the reinforcement comparison strategy. The results demonstrate that in the investigated model, total revenue and total payoff of the SU operator (auctioneer) and SU terminals (bidders) are characterized by the existence of the global optimum, thus there exists the optimal sensing time guaranteeing the optimum economic factors for both SU operator and SU terminals