This work proposes a novel approach for the automatic generation and tuning of complete Takagi-Sugeno-Kang fuzzy rule based systems. The examined system aims to explore the effects of a reduced search space for a genetic programming framework by means of grammar guidance that describes candidate structures of fuzzy rule based systems. The presented approach applies context-free grammars to generate individuals and evolve solutions through the search process of the algorithm. A multi-population approach is adopted for the genetic programming system, in order to increase the depth of the search process. Two candidate grammars are examined in one regression problem and one system identification task. Preliminary results are included and discussion proposes further research directions

Gabrys, Bogdan

Tsakonas, Athanasios

English

Bournemouth University Research Online

EVOLVING TAKAGI-SUGENO-KANG FUZZY SYSTEMS USINGMULTI POPULATION GRAMMAR-GUIDED GENETICPROGRAMMINGAthanasios Tsakonas, Bogdan GabrysSmart Technology Research Center, Dept. of Design, Engineering and Computing, Bournemouth University, Poole, UKatsakonas@bournemouth.ac.uk, bgabrys@bournemouth.ac.ukKeywords: Genetic Programming, Fuzzy Rule Based Systems, Evolutionary ComputationAbstract: This work proposes a novel approach for the automatic generation and tuning of complete Takagi-Sugeno-Kang fuzzy rule based systems. The examined system aims to explore the effects of a reduced search spacefor a genetic programming framework by means of grammar guidance that describes candidate structures offuzzy rule based systems. The presented approach applies context-free grammars to generate individuals andevolve solutions through the search process of the algorithm. A multi-population approach is adopted for thegenetic programming system, in order to increase the depth of the search process. Two candidate grammarsare examined in one regression problem and one system identification task. Preliminary results are includedand discussion proposes further research directions.1 INTRODUCTIONThe application of fuzzy rule-based systems (FRBS)has been proven effective in a wide area of domainsand problem tasks (Jang 97). Among their implemen-tations, Mamdani-FRBS have been used for classi-fication tasks, while regression and system identifi-cation application areas have been dominated by theTakagi-Sugeno-Kang (TSK) -FRBS. In order to ex-ploit the power of an FRBS, commonly a trainingphase takes place with the neuro-fuzzy techniquesmost often used. In their simplest approach, thesesystems require the pre-determination of the rule-basesize. Advanced approaches may adopt an ad-hoc orheuristic incremental process to generate the fuzzyrule base. Others make use of evolutionary techniquesaiming to provide efficient rule-base generation.Genetic programming (GP) is a successful branchof evolutionary computation effectively applied in awide range of tasks such as symbolic regression andnetwork design (Koza 92). In the past, GP has beensuccessfully tested for the production of Mamdani-FRBS (Alba et al. 96). Consequently, the perspec-tive for generating TSK-FRBS within GP frameworkhas always been regarded as a promising researchfield, since such a system could combine the attrac-tive properties of both methods.In response to the need for such a hybrid, GPwas effectively combined with TSK-FRBS in the past.These approaches commonly involved partial contri-bution of GP techniques, such as the assistance by GPin locating proper membership functions for a TSK-FRBS (Hoffman and Nelles 01) or the co-evolutionof several segments of a TSK-FRBS (Delgado et al.04). In this paper, we propose an integrated approach,for the automatic design and tuning of complete TSK-FRBS for regression and system identification. To ac-complish this, we use grammars to describe the com-plete structure of TSK-FRBS within a GP individual,thus mapping one GP-individual to exactly one TSK-FRBS. Using this approach we are able to generatearbitrarily sized TSK-FRBS which can be tuned usingeasy to formulate and implement integrated approach.We furthermore enhance our search procedure by in-corporating multi-population architectures to the GPsolution pool. We test two variants of this system inone prediction and one system identification task.The content of this paper is organised as follows.Next section describes the scientific background ofthe methods to be used. In Section 3, the design ofthe system and its implementation are presented. Theresults and discussion related to the system effective-ness are included in Section 4. The conclusions andsuggestions for further research are drawn in the Sec-tion 5.2 BACKGROUNDFuzzy sets are an extension to the classic (crisp) sets,where the transition for a value from belonging to aset and not belonging to the set is gradual and quanti-fied by a suitably chosen membership function. Whenperforming fuzzy reasoning, we apply, in a general-ized form, the traditional two-valued logic, the modusponens. More specifically, a fact can be more or lesstrue, based on the truth of another fact. The generalexpression of a rule in a Takagi-Sugeno-Kang (TSK)fuzzy rule based system has the following form (Jang98):Ri : i f x1 is dk and ... then y= f (x1, ...,xn)with C (1)(i = 1,2, ...,m),(x1, ...,xn ∈ X), C ∈ [0,1]In this formula, C is the certainty factor and X is theinput set.Genetic programming - GP (Koza 92) is a searchmethodology belonging to the evolutionary computa-tion family, which in its canonical form permits theautomatic generation of programs. Among success-ful evolutionary computation approaches, GP keeps asignificant position due to its beneficial qualities, suchas the production of arbitrary-sized solutions and theupholding of population diversity throughout runs.Multi population evolutionary models, or islandmodels, divide the population into sub-populations,often called demes, and periodically exchange a num-ber of individuals, simulating migration. In this pa-per, we adopt a five island model, and we employ acanonical setup, according to literature (Ferna´ndez etal. 03).In canonical GP, the type of nodes is implicitly de-fined by the number of each node’s arguments (Koza92). Hence, usually there are two types of nodes in thestandard GP: a) terminals - non-function nodes; andb) functions - non-terminal nodes. In order to producein an efficient way more complex structures withinGP, such as fuzzy rule based systems, we must definea stricter hierarchy of the potential architectures. Toaccomplish this, one of the most efficient ways is theapplication of a context-free grammar (Montana 95).The standard notation to express context-free gram-mars is in the Backus Naur Form (BNF).The idea to incorporate fuzzy inference into GPis not new. In (Alba et al. 96) a model that uses acontext-free grammar for the production of Mamdanifuzzy rule bases has been examined. The system hasproven competitive in various classification tasks. In(Delgado et al. 04), a genetic algorithm is proposedfor a co-evolutionary system that produces TSK fuzzysystems. In (Chen 07), the generation of a specificclass of TSK fuzzy systems is accomplished usingevolutionary programming. In (Hoffman and Nelles01), a genetic programming system has been used toimprove a greedy algorithm (LOLIMOT) in search ofdata clusters by partitioning the search space.Our approach aims at the production of TSK fuzzyrule based systems in the context of multi popula-tion grammar guided genetic programming. It hasseveral advantages in comparison to the previous at-tempts. Firstly, it makes use of the genetic program-ming framework which has a number of desirableproperties in terms of maintaining population diver-sity and providing arbitrary solution size. Secondly,our approach is comparatively simple to implement,since only one population is maintained, and all fuzzyrule base parameters are tuned within one system run.Finally, it is an integrated approach, that generalizesthe applicability of GP for the production of fuzzysystems by generating complete TSK rule bases forregression and system identification. The structure ofthe TSK fuzzy system is fully described within the GPfunctions, and every tuning parameter is computed byevolutionary means.3 DESIGN ANDIMPLEMENTATIONWe have evaluated the proposed system within twodata domains. The first domain is the Concrete SlumpTest (Yeh 08). We have retained 20% of the data astest set with the remaining 80% used for learning. Thesecond problem addressed in this work is a systemidentification task. We considered the Mackey-Glassnon-linear chaotic time series, as this is commonlyused as a test for TSK models (Chen 07). We gener-ated 400 data points out of which 80 were were usedas the test set with the remaining data used for train-ing.We examined two system configurations. The firstsystem evolves a rule base with the rules in the follow-ing form:Ri : i f x1 is µk1 [and x2 is µk2 ...]then y = f (xc) with C (2)f (xc) = w0xnc +w1xn−1c + ...+wn−1xc+wn (3)(i = 1, ...,m),(x1, ...,xn ∈ X), C ∈ [0,1]The second system evolves a rule base with the rulesin the following form:Ri : i f x1 is µk1 [and x2 is µk2 ...]then y =f (xc1) f (xc2)f (xc3) f (xc4)+ f (xc5) with C (4)f (xcr) = w0xncr +w1xn−1cr + ...+wn−1xcr +wn (5)(i = 1, ...,m),(x1, ...,xcn ∈ X), C ∈ [0,1]where C is the certainty factor, X is the input set, µknare membership functions, and w1, ...,wn are arith-metic expressions calculated during evolution. Forexpressing the fuzzy relations, Gaussian membershipfunctions (MF) were selected. In this paper, we se-lected to use 3 MFs per attribute. Tuning of theseMFs was also applied by evolutionary means. The se-lection of the variables in each rule is also guided byevolution. As fitness function, the mean square er-ror (MSE) was selected. Table 1 summarizes the GPtraining parameters.Table 1: Genetic programming training parameters.Parameter ValueGP System Grammar-Guided GPIslands 5Island topology RingIsolation time 50 generationsMigrants number 10 individualsMigrants type Elite individualsTotal population 1,000 individualsSelection TournamentTournament size 7Elitism YesCrossover rate 0.7Mutation rate 0.3Max.individual size 4,000 nodesMax.generations 5004 RESULTS AND DISCUSSIONOur test results for the P1 and P2 systems in the Con-crete Slump Test problem are shown in Table 2. Exactreconstruction of the test data set was not possible, asthere is not related information in the literature (Yeh08), however we selected to apply the same numberof randomly chosen test records. Our results, in thisexample, are given in terms of Pearson correlationcoefficient, to allow direct comparison with existingliterature results. The highest reported accuracy inprevious research, for this data set is R2=0.922 (Yeh08) where a neural network model was applied.An example part of the derived rule base of the P1system has the following form:if Cement is Low then y = 1.4x2+1.5x−9.3,(x: Water)if Cement is Medium then y =−2.3x2−0.2x−8.1,Table 2: Results for the Concrete Slump Test.Parameter P1 P2Size (nodes) 2437 410Size (rules) 202 9R2 0.9127 0.80794(x: Coarse Aggr.)if Water is Medium then y =−1.4x2−1.7x−3.3,(x: Fly Ash)As it can be seen, in this problem, the P1 systemcompares well with the results reported in the liter-ature, and in addition the output format allows foreasier interpretation of the results. Although the P2system achieved similar to P1 scores in our trainingand validation set, its performance for the test set waslower.The test results for P1 and P2 systems for theMackey-Glass data are summarised in Table 3.Table 3: Results for the Mackey-Glass time series.Parameter P1 P2Size (nodes) 1867 457Size (rules) 98 8MSE 0.0033591 0.001336As expected, the higher order P2 model was ableto provide a better approximation to this time-series.As an example output, one of the rules of the acquiredsolution by P2 system has the following form:if x4 is Low then y =f1(x3) f2(x3)f3(x4) f4(x3)+ f5(x2) where:f1(x3) = 1.14x43+0.15x33+0.44x23+0.24x3−0.23,f2(x3)=−1.24x43+0.15x33+0.719x23+1.23x3+0.73,f3(x4) = 0.42x44−0.36x34−1.22x24−0.74x4+1.11,f4(x3) = 4.61x43−0.8x33+1.28x23−0.63x3−0.25,f5(x2) =−1.17x42 +1.05x32 +0.33x22−0.63x2+0.37.(x2 : Slag, x3 : Fly Ash, x4 : Water).Although direct comparison with previous literatureresults is not possible due to inability to reconstructthe exact training data set, past research is shown, forreference reasons, in Table 4. As it can be seen inthe table, our approach compares well to other modelsthat do not employ local search methods.5 CONCLUSION AND FURTHERRESEARCHThis paper presented a system for the generation ofTakagi-Sugeno-Kang fuzzy rule based systems for re-gression and system identification, by means of ge-netic programming. The proposed approach carriesseveral advantages over past related research. Firstly,Table 4: Results for the MacKey-Glass time series. Regres-sion scores from (Jang 97), (Kim and Kim 97), (Wang 92)and (Lee and Kim 94).System RMSELinear regression model 0.55Auto regressive model 0.19Sixth order polynomial 0.04Back propagation NN 0.02GA and fuzzy system (5 MFs) 0.049206GA and fuzzy system (7 MFs) 0.042275GA and fuzzy system (9 MFs) 0.037873Wang Product T-norm 0.907Wang Min T-norm 0.904ANFIS 0.007P2-TSK-GP (3 MFs) - this paper 0.036548the system is capable to automatically produce arbi-trarily large and complex TSK fuzzy systems, accord-ing to the needs of a specific problem. Secondly, itprovides flexibility in the selection of non-linear func-tions fired per rule. Finally, the output model is inter-pretable by humans in contrast to some other modelslike MLPs, since it is in the form of fuzzy rules. In thispaper we have presented preliminary results of the ex-periments which while focusing on certain character-istics and capabilities of the GP algorithms producedencouraging results warranting further investigations.Further research will be primarily focused on anadvanced grammar design for efficient combinationsof polynomials. Increasing the number of the mem-bership functions is also expected to improve the ac-curacy of the system. Integration with ensemble sys-tems will also be considered.ACKNOWLEDGEMENTSThe research leading to these results has receivedfunding from the European Commission within theMarie Curie Industry and Academia Partnerships andPathways (IAPP) programme under grant agreementn. 251617.REFERENCESAlba E., Cotta C., Troya J.M., Evolutionary Design ofFuzzy Logic Controllers Using Strongly-TypedGP, In Proc. 1996 IEEE Int’l Symposium onIntelligent Control New York, NY., 127-132,1996.Augusto D.A., Barbosa H.J.C., Ebecken N.F.F., Co-evolutionary Multi-population Genetic Program-ming for Data Classification GECCO’10 , July7-11, 2010, Portland, USA.Blake C. L., Merz C. J., UCI repository of machinelearning databases, http://www.ics.uci.edu/mlearn/ML-Repository.html, 2001.Chen Y., Yang B., Abraham A., Peng L., Au-tomatic Design of Hierarchical Takagi-SugenoType Fuzzy Systems Using Evolutionary Algo-rithms IEEE Trans. on Fuzzy Systems , 15:3,2007.Delgado M.R., Zuben F.V., Gomide F. , Coevolution-ary genetic fuzzy systems: a hierarchical collab-orative approach Fuzzy Sets and Systems , Else-vier, 141, 89-106, 2004.Ferna´ndez F., Tommassini M.,Vanneschi L., An Em-pirical Study of Multipopulation Genetic Pro-gramming Genetic Programming and EvolvableMachines 4:1, Kluwer, 2003.Hoffman F., Nelles O., Genetic Programming forModel Selection of TSK-Fuzzy Systems Infor-mation Sciences, Elsevier, 136, 7-28, 2001.Jang J-S. R., Sun C.T., Mizutani E.,Neuro-Fuzzy andSoft Computing , Prentice Hall, NJ, 1997.Jang J-S. R., Neuro-fuzzy modeling for nonlinear dy-namic system identification in: Ruspini E. H. ,Bonissone P. P. , Pedrycz W., eds., Handbookof fuzzy computation , Institute of Physics Pub-lishing, Dirak House, Temple Back, Bristol, UK1998.Kim D., Kim C., Forecasting time series with geneticfuzzy predictor ensemble,IEEE Trans. FuzzySyst., 5:4, 523-535, 1997.Koza J.R. Genetic programming - On the program-ming of computers by means of natural selec-tion The MIT Press, Cambridge, Massachussets,USA, 1992.Lee S.-H., Kim I., Time series analysis using fuzzylearning, inProc. Int’l Conf. Neural Inform. Pro-cess., Vol. 6, Seoul, Korea, 1577-1582, 1994.Manderick B., Spiessens P., Fine-Grained ParallelGenetic Algorithms in Proc. of the 3rd Int’lConf. on Genetic Algorithms, Morgan Kauf-mann, 1989.Montana D.J., Strongly Typed Genetic Programming,Evolutionary Computation, 3:2, 1995.Wang L.X., Mendel J.M.,Generating fuzzy rules bylearning from examples,IEEE Trans. Systems,Man and Cybernetics, 22:6, 1414-1427, 1992.Yeh, I-C., Modeling slump of concrete with fly ashand superplasticizer, Computers and Concrete,5:6, 559-572, 2008.

An Empirical Study

Automatic Design of Hierarchical Takagi-Sugeno Type Fuzzy Systems Using Evolutionary Algorithms

Coevolutionary genetic fuzzy systems: a hierarchical collaborative approach Fuzzy Sets and Systems ,

Evolutionary Design of Fuzzy Logic Controllers Using Strongly-Typed GP,

Fine-Grained Parallel Genetic Algorithms in

Forecasting time series with genetic fuzzy predictor ensemble,IEEE Trans.

Genetic programming - On the programming of computers by means of natural selection

Genetic programming - On the programming of computers by means of natural selection TheMITPress,

Genetic Programming for Model Selection of TSK-Fuzzy

Mendel J.M.,Generating fuzzy rules by learning from

Modeling slump of concrete with ﬂy ash and superplasticizer,

Neuro-fuzzy modeling for nonlinear dynamic system identiﬁcation in:

Strongly Typed Genetic Programming,

Time series analysis using fuzzy learning,

UCI repository of machine learning databases, http://www.ics.uci.edu/ mlearn/ML-Repository.html,

Evolving Takagi-Sugeno-Kang fuzzy systems using multi-population grammar guided genetic programming

Evolving Takagi-Sugeno-Kang fuzzy systems using multi-population grammar guided genetic programming

Abstract

Similar works

Full text

Available Versions

Bournemouth University Research Online