Classical Machine Learning methods are usually developed to work in static data sets. Yet, real world data typically changes over time and there is the need to develop novel adaptive learning algorithms. In this work, a number of algorithms, combining Neural Network learning models and Evolutionary Computation optimization techniques, are compared, being held several simulations based on artificial and real world problems. The results favor the combination of evolution and lifetime learning according to the Baldwin effect framework

Cortez, Paulo

Neves, José

Rocha, Miguel

WSEAS TRANSACTIONS ON SYSTEMS

Universidade do Minho: RepositoriUM

Evolutionary Neural Network Learning Algorithmsfor Changing Environments∗MIGUEL ROCHA1, PAULO CORTEZ2 and JOS ´E NEVES11Departamento de Informa´tica / 2Dep. Sistemas de Informac¸a˜oUniversidade do Minho14710-057 Braga / 24800-058 Guimara˜esPORTUGALmrocha@di.uminho.pt pcortez@dsi.uminho.pt jneves@di.uminho.pthttp://www.di.uminho.pt/˜pcortez/ http://www.dsi.uminho.pt/˜jneves/Abstract: Classical Machine Learning methods are usually developed to work in static data sets. Yet, real worlddata typically changes over time and there is the need to develop novel adaptive learning algorithms. In this work,a number of algorithms, combining Neural Network learning models and Evolutionary Computation optimizationtechniques, are compared, being held several simulations based on artificial and real world problems. The resultsfavor the combination of evolution and lifetime learning according to the Baldwin effect framework.Key-Words: Baldwinian and Lamarckian Effects, Evolutionary Programming, Multilayer Perceptrons.1 IntroductionIn recent years, there has been a remarkable devel-opment of the Data Mining and Machine Learning(ML) arenas, which aim at the extraction of patterns ormodels from observed data. This interest arose due toan ever-increasing load of data, which often presentshigh complexity, while human experts are limited andmay overlook important details. Yet, the majority ofthese techniques use static off-line learning (where thedata is stored and accessed repeatedly), while in manyapplications data is rather changing and evolving [5].In these cases, there is a clear need for adaptive learn-ing, where the learning function changes over time.On the other hand, living creatures have survivedin hazardous and dynamic environments for millionsof years, by taking advantage of two interacting pro-cesses: evolution and lifetime learning. Evolution is aslow process that takes place at the population level,determining the basic structure of an organism, whilelifetime learning is responsible for the adaptation atthe individual’s level. In computational terms, thesecan be materialized via the Evolutionary and NeuralComputation fields. The former is suitable for globalsearch and the latter for fine tuning (local search).When combining evolution and learning, two ma-∗This work was supported by the FCT projectPOSI/ROBO/43904/2002 (partially funded by FEDER).jor frameworks can be addressed, namely the Lamar-ckian [4] and Baldwinian strategies [1], which differon the fact that acquired traits are recoded into thechromosome or not. Although there has been pastwork in this arena, most of the studies consider onlya subset of the possible strategies or focus mostly onthe benefits of lifetime learning, being the trade-offbetween costs and profits rarely considered [9].In this work, the evolution process is approachedvia Evolutionary Programming (EP), where each in-dividual encodes the weights of a MultiLayer Percep-tron (MLP) [8]. Lifetime learning is addressed by thetraining of the MLP, using a gradient descent basedprocedure [7]. Five different strategies will be testedin several classification tasks, being the comparisonsbased on computation time, so that they can be fairer.2 Changing EnvironmentsThe experiments that will be considered in this workendorse an important ML category, classificationtasks [5], where the aim is to label a specific data itemto a categorical class. In this study, two artificial andtwo real world tasks were selected:6 Bit Classification (6BC) - Set by 26 binary patternsof 6 inputs and 1 output, whose value is set to 1if the number of true input bits is odd [7].BEGINSelect P initial examples (from the database)FOR ( each event at time t ∈ {t1, . . . , tN} ) DODiscard E examplesSelect new E examples (from the database)ENDFig. 1: Pseudo-code for the dynamic environments.Three Color Cube (TCC) - The painting of a 3Dcube made up by a 3x3x3 grid of blocks [4]. Theinputs stand for the blocks’ coordinates on theX , Y and Z axis and the output denotes a color(the corners are black, the cubes in the center arewhite, the other are gray).Contraceptive Method Choice (CMC) - Indonesiancontraceptive method adoption, based on a num-ber of socio-economical attributes (e.g. wife’sage) [3]. The data has 1473 instances with 9 in-puts (2 continuous, 4 categorical and 3 binary)and 3 choices (no use, long-term or short-term).Car Evaluation Database (CEB) - The evaluation ofcars (4 labels) according to price, technical andcomfort aspects, with a total of 1728 examplesand 6 nominal inputs [3].Although these tasks are static, is is possible to sim-ulate changing environments using the dynamic pro-cess described in Figure 1, which requires the defini-tion of the following parameters: P , the number ofpatterns to be fed to the learning algorithm in eachperiod; E, the number of examples that are changedat each event; and t1, . . . , tN , the set of time eventsthat will induce changes. The last two factors willinfluence the type of environment: soft changing (orconcept drift), when changes occur gradually; andhard changing (or concept shift), when changes occurabruptly. Both types of environments will be tested:Soft (S) - one pattern will change at each second(E = 1, t ∈ {1s, 2s, 3s, . . .}); andHard (H) - several patterns will be commuted over awider period (ti+1 − ti = 20s for the artificialtasks and 50s for real ones).Since the artificial tasks (6BC and TCC) presentfew examples, these were adapted according to:6BCS - the first output patterns (P = 64) are ran-domly set; then, during each event, one output isselected to change (from 0 to 1 or the reverse);Table 1: The MLP topologies.Task Inputs Hidden Outputs Weights6BP 6 6 1 49TCC 3 8 3 59CMC 21 10 3 253CEB 21 10 4 2646BCH - the 6BC task, except that all the desired out-puts (E = P = 64) will flip periodically;TCCS - each cube is initially assigned a randomcolor, and then, at each event, the color of onerandomly chosen cube is changed;TCCH - the static rules prevail, although the cubesare periodically repainted (E = P = 27), fol-lowing a predefined order (black follows gray,gray follows white and white follows black).For the tasks CMC and CEB, the parameters wereset to E=1, P=300 (S); and E=50, P=300 (H).3 Learning AlgorithmsMulti-Layer Perceptrons (MLPs) are one of the mostpopular Neural Networks, where neurons are groupedin layers and only forward connections exist [2].MLPs are appealing due to their capability to modelcomplex multi-dimensional data, even when noise ispresent, often outperforming other ML techniques [6].In this work, fixed fully connected MLPs wereadopted, with one hidden layer, bias connections andlogistic activation functions. Before learning, the in-put (output) values were rescaled to the range [−1, 1]([0, 1]), using a 1-of-C encoding scheme for the nom-inal attributes (with more than two possible classes).The topologies were chosen to make the learning pos-sible with a minimum complexity (Table 3).Five distinct algorithms will be defined to approacheach learning task:The Connectionist Model (CM). Under these cir-cumstances, the learning is achieved by a singleMLP and the training performed by the RPROPalgorithm [7].The Population of Connectionist Models (PM). Aset of 20 MLP individuals will improve onlyvia the learning algorithm (RPROP); i.e., noreproduction or selection procedure is applied.......    ...CrossoverMutationSelectionLearningPopulationDecode BaldwinianLamarckianEncodeRPROPFig. 2: The Baldwinian and Lamarckian strategies.The Darwinian Model (DM). The learning processis accomplished by EP, where a population of20 real-valued chromosomes is evolving, eachcoding the weights of a MLP [8]. In each iter-ation, 50% of the individuals are kept from theprevious generation, being the remaining bredthrough the application of a gaussian mutation,adding a small perturbation to a variable numberof genes (this value is randomly set between 1and 20% of the total number of weights).The Lamarckian Model (LM). The LM combinesboth lifetime learning and evolutionary ap-proaches. EP is still used, although the muta-tion operator is replaced by a random one, whichreplaces a number of weights by values in therange [−1.0, 1.0]. In previous work, this macro-mutation strategy has achieved better results [8].Each individual is allowed to learn during itslifetime, by running the RPROP algorithm for50 epochs in each generation of the EP process.Then, the improved weights are encoded backinto the chromosome (Figure 2).The Baldwinian Model (BM). Similar to the LM,except that lifetime learning is only used to im-prove the fitness of the individuals, and the newweights are not encoded back into the genome(Figure 2). This means that, in the process of re-production, the offspring does not inherit the ac-quired genetic information from their ancestors.For all models, the initial weights are randomly setwithin the range [−1.0; 1.0]. The accuracy is mea-sured by the Root Mean Squared Error (RMSE):RMSE =√∑Pi=1∑Oj=1(Ti,j−MLPi,j)2PO(1)where P denotes the number of the training patterns;O the number of the MLP outputs; Ti,j the target andMLPi,j the network value; both for the output j andthe i input pattern. This metric is used as the fitnessvalue in the evolutionary approaches. For the popula-tion based models (PM, DM, LM and BM), the overallaccuracy is given by the RMSE of the best individual.4 ResultsAll experiments were conducted using Java program-ming environments, running on a PC with a PentiumV 2.4 GHz processor. The results were comparedin terms of two parameters: the overall learning’saccuracy (RMSE), and the process’ efficiency (timeelapsed, in seconds). For all models, at each time slot,one considers the average of the results obtained in20 independent runs. Finally, the termination criteriawas set by a time limit T , here set to T = 1000s.Regarding the soft changing experiments (Figure5), the CM is incapable of escaping the initial learningbias, presenting the worst performance. Both PM andDM show stable patterns, but seem unable to improve.When combining evolution and learning better resultsare achieved, being the BM the best alternative.For the hard changing tasks, only the best strate-gies (BM and LM) were plotted, to simplify the vi-sual analysis (Figure 6). Here, the error ranges arehigher for the artificial tasks, which is explained bythe larger number of commuted patterns. Again, anddespite the more radical changes, the BM still man-ages to get a steady performance over time, being thebest solution.To achieve a global accuracy, the values within thefirst 200 seconds were discarded (a start-up time setto allow the tunning of each model); then, a box plotwas computed over the remaining values, includingthe statistics (from bottom to top): minimum, meanminus standard deviation, mean, mean plus standarddeviation and maximum (Figures 3 and 4). Resultsconfirm that the BM presents the lowest mean val-ues. Although the standard deviation ranges overlap,it must be noticed that the 95% confidence intervalsand paired t-student tests show significant differences.5 Conclusions and Future WorkThe results obtained support the idea that the com-bination of evolution and learning is an interestingapproach, when facing changing environments. Al-though the simulated environments are commuted pe-riodically, this information is not used by the pro-posed learning algorithms, which is useful whenchanging events are hard to detect. Even whenchanges can be detected, sometimes these are so rapidthat it is not possible to rebuild the learning model.Referring to the LM vs BM debate, this work sup-ports the latter when facing changes. One explanationmay be that the BM is evolving not a final solution, butinstead a good set of initial weights.In the future it is intended to enlarge the exper-iments to real world domains (e.g. intensive careunits) and other problem types (e.g. reinforcementlearning). Furthermore, the learning algorithms couldbe adapted to perform a simultaneous evolution ofneural architectures and weights.References:[1] J.M. Baldwin. A New Factor in Evolution. Amer-ican Naturalist, (30):441–451, 1896.[2] C. Bishop. Neural Networks for Pattern Recogni-tion. Oxford University Press, 1995.[3] C. Blake and C. Merz. UCI Repository of Ma-chine Learning Databases, 1998.[4] P. Cortez, M. Rocha, and J. Neves. A LamarckianApproach for Neural Network Training. NeuralProcessing Letters, 15(2):105–116, April 2002.[5] M. Goebel and L. Gruenwald. A Survey ofData Mining and Knowledge Discovery SoftwareTools. SIGKDD Explorations, 1(1):20–33, 1999.[6] T. Lim, W. Loh, and Y. Shih. A Comparisonof Prediction Accuracy, Complexity and TrainingTime of Thirty-three Old and New ClassificationAlgorithms. Machine Learning, 40(3):203–228,2000.[7] M. Riedmiller. Supervised Learning in Mul-tilayer Perceptrons - from Backpropagation toAdaptive Learning Techniques. Computer Stan-dards and Interfaces, 16, 1994.[8] M. Rocha, P. Cortez, and J. Neves. Evolution-ary Neural Network Learning. In F. Pires andS. Abreu, editors, EPIA 2003 Proceedings, LNAI2902, pages 24–28. Springer, 2003.[9] P. Turney. Myths and Legends of the BaldwinEffect. In Proc. of ICML96, pages 135–142.Springer, 1996.CM PM DM LM BM0.20.30.40.50.60.7CM PM DM LM BM0.20.30.40.50.6CM PM DM LM BM0.300.350.400.45CM PM DM LM BM0.000.050.100.15Fig. 3: Box plots for the soft experiments (6BCS ,TCCS , CMCS and CEBS).CM PM DM LM BM0.20.30.40.50.60.70.8CM PM DM LM BM0.20.30.40.50.60.7CM PM DM LM BM0.300.350.400.45CM PM DM LM BM0.000.050.100.150.20Fig. 4: Box plots for the hard experiments (6BCH ,TCCH , CMCH and CEBH ). 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)CMPMDMLMBM 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)CMPMDMLMBM 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)CMPMDMLMBM 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)CMPMDMLMBM 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 500  550  600  650  700  750  800  850  900  950Fig. 5: Results for soft experiments (6BCS , TCCS , CMCS and CEBS). 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)LMBM 0.2 0.3 0.4 0.5 0.6 0.7 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)LMBM 0.3 0.35 0.4 0.45 0.5 0.55 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)LMBM 0.28 0.29 0.3 0.31 0.32 0.33 0.34 0.35 0.36 0.37 500  550  600  650  700  750  800  850  900  950 0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.1 0  100  200  300  400  500  600  700  800  900Error (RMSE)Time (seconds)LMBMFig. 6: Results for the hard simulations (6BCH , TCCH , CMCH and CEBH ).

Evolutionary neural network learning algorithms for changing environments

http://repositorium.sdum.uminho.pt/bitstream/1822/426/1/483-229.pdf

Evolutionary neural network learning algorithms for changing environments

Abstract

Similar works

Full text

Available Versions

Universidade do Minho: RepositoriUM