Fuzzy logic is widely applied in control and modeling for its robustness, simplicity and clarity. It is also applied in classifier design with rules directly generated from numerical data. Some available rule generation methods, however, are either too complicated to implement or impractical for high dimensions. In this paper, we propose a new fuzzy classifier architecture. At the very beginning the training data is clustered at the input space. Fuzzy sets are then defined based on these clusters with triangular membership function. The outputs in the rule conclusion are initially determined by the “normalized vote” in the corresponding cluster. Fuzzy sets and conclusions can be adjusted through training. The proposed fuzzy system is simple in structure, and can be fast trained and easily implemented. Its classification performance is generally better than artificial neural network.published_or_final_versio

Chan, FHY

Hung, N

Lam, FK

Yang, YS

HKU Scholars Hub

Title A new fuzzy classifier with triangular membership functionsAuthor(s) Yang, YS; Chan, FHY; Lam, FK; Hung, NCitation International Conference on Neural Networks Proceedings,Houston, TX, 9-12 June 1997, v. 1, p. 479-484Issued Date 1997URL http://hdl.handle.net/10722/46013Rights©1997 IEEE. Personal use of this material is permitted. However,permission to reprint/republish this material for advertising orpromotional purposes or for creating new collective works forresale or redistribution to servers or lists, or to reuse anycopyrighted component of this work in other works must beobtained from the IEEE.A NEW FUZZY CLASSIFIER WITH TRIANGULAR MEMBERSHIP FUNCTICINS ‘Yong-Sheng Yang ‘Francis H. Y. Chan ‘F. K. Lam 2 Hung Nguyen Dept. of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong School of Electrical Engineering, University of Technology, Sydney, NS W 2007, Australia ‘ysyang@hkueee.hku.hk, ‘fiychan@h]kueee.hku.hk, ‘fklam@hkueee.hku,hk, *htn@ee.uts.edu.au 1 2 ABSTRACT Fuzzy logic is widely applied in control and modeling for its robustness, simplicity and clarity. It is also applied in classifier design with rules directly generated from numerical data. Some available rule generation methods, however, are either too complicated to implement or impractical for high dimensions. In this paper, we propose a new fuzzy classifier architecture. At the very beginning the training data is clustered at the input space. Fuzzy sets are then defmed based on these clusters with triangular membership function. The outputs in the rule conclusion are initially determined by the “normalized vote” in the corresponding cluster. Fuzzy sets and conclusions can be adjusted through training. The proposed fuzzy system is simple in structure, and can be fast trained and easily implemented. Its classification performance is generally better than artificial neural network. I. INTRODUCTION In recent years a lot of research was done on fuzzy modeling and fuzzy classifier [ 11-[SI which generates rules from numerical data. Many methods cut the input space into fuzzy hyper-boxes by separating each input component so that it becomes impractical in case of too many inputs. Some methods are so complex in architecture [3] that they are only advantageous for large scale system. In this paper, we propose a new fuzzy logic classifier which can be easily trained and implemented while keeping satisfactory classification performance. In input space the training data is clustered into hyper-spheres which are thought to be usually more appropriate than the hyper-boxes in reflecting the pattern distributions. Enlightened by [9], we then generate rules from each cluster. The rules can be tuned through training. It is an improved version of the fuzzy classifier presented in [ 141. The improvement exists in, that a technology called “normalized vote” is oiriginally introduced so that the classifier requires very little training before being adjusted to optimal parameter setting, and that triangular membership functions (MF) are applied in place of Gaussian-shaped MF so tlhat the classifier is able to run faster and readily implemented on microprocessors or fuzzy chips. Section I1 introduces the formation and training of the proposed fuzzy classifier. Section 111 presents two application examples verifying its better performance. Section IV contains some further discussion. And the system is then concluded in section IV. 11. ALGORITHM Our aim is to design a classifier which applies fuzzy inference so that it works like the way of human being. Fuzzy sets are to be defined on each dimension of the input data space. The classifier will work on many rules like: IF X’(X,, X, ,..., X,)EFk, i.e., XI€Fkl and Xz€l’kz and ... and X,EFkn THEN xEClassj where (x,, xz, ..., x,) is the n-dimensional iinput, Fk =Fk,xF,x ... xFk, is the fuzzy set in the premise of the k-th rule, and j=1,2 ...., m (m is the total number of classes). To quantifi, the rule’s conclusion, we define an output vector wk=(Wklyw@y..  wh:), where wk, is the degree of how much an input belongs to class j by a specific rule; 0 1  wkj 11. In fact we have extended the crisp class to kzzy class thralugh the adoption of the output vector. Based on this idea, tlhe fuzzy set’s centroid and width will be initialized by the results out of Basic ISO-DATA algorithm [LO]. Suppose M clusters: C,, C,, ... , CMr are generated from ISO-DATA with ( ~ ~ , x u  ,..., xkn)as (2k’s centroid vector and (Ok,,Ok,,. . . p k J  as (2,’s radius vector, where - -  - 0-7803-4122-8/97 $10.0001997 IEEE 419 k=1, ..., M, and radius vector being equal in each dimension. M fuzzy sets: F,, F,, ..., F,, are then defined based on the M clusters respectively by the triangular- product membership function pF (.) as: Given n- dimensional input x=(x,, x,, . . . , XJ, the degree of XEF, is k where P k i ( x i ) =  - - - ( x i  - X k i  + 3 0 k i ) / ( 3 0 k i )  for Xki - 3 0 k i  <x i  < x k i  ( - x i  + Xki  + 3 0 , ; )  / (30,;) for - - - Xki < xi 2 Xki + 3 0 k i  ( I )  0 else and i=l, ..., n; k=l, ..., M. Fig. 1 illustrates an exemplar fuzzy set definition via clusters in two-dimensional input space. M rules are designed respectively corresponding to the M clusters as: If x belongs to fuzzy set F,, then outputs are wk= (w,,, wk2, . . .? Wkm) Where wkt is yielded from the “normalized vote” in cluster k: if there is vkt items in the cluster belongs to class t, then Wkt =m ‘kt , fort=l, ..., m; k=l, ... M. - k V k s  s=l  o=(ol,02,. .,om), the final outputs after the M product- inference rules and centroid average defuzzifier, are given by: ~r 1 8 kt ki ( x i  11 O t  = fort=1,2 ,..., m (2) , k=l L i=l  J The sample is then assigned to the class with maximum output among the m values. The network representation of the fuzzy classifier is demonstrated in Fig. 2, To make the fuzzy system work more precisely, we show how rules and membership functions are adjusted by the error back-propagation method. For N input-output pairs: (x(’),y(’)), (x(’),y(’)),. . ., (xm),y”)), mean square error is defied as: Calculating the partial differentiation of (3) with respect to w k , x k i , o k i  yields: - (4) ( 5 )  for k=l, ..., M; i=l, ..., n; t=l, ..., m. Hence W k t  , Xki , ob can be trained by the steepest-descent method to reduce E: - where a=O.OO1-0.1 is the step-size. 111. EXPERIMENTAL RESULTS Initially, the performance of this fuzzy classifier is verified with the use of a set of synthesizes overlapping data. Then it is applied to the functional electromyogram (EMG) classification for prosthesis control [I  11. For comparison purpose, a three-layer artificial neural network classifier [ 121 is also developed using the same data sets. 480 A. Synthetic data classijkation Two Gaussian distributed classes are generated in the two-dimensional space with same variances cr in each dimension. Their centers are located at (1,l) and (2,2) respectively. 200 samples are generated for each class. Half the samples are used for training, and the left for testing. We defme CSEP, the separation coefficient, to reflect the class overlap: d 0.236 --  3 * ( ( 0 + 0 )  cr ‘SE, = where d= f i  is the class center distance. Thirteen clusters are generated by ISO-DATA according to experiments, and hence there are thirteen rules. There are two outputs in each rule corresponding to the two classes. With relation to the separation coefficient, the classifiers’ training mean square errors and classification error rates are listed in table I. Generally speaking, the fuzzy classifier works better than ANN with a little decrement in error rate. In addition it is noticed that fuzzy classifier needs very few training epochs. B. EMG data classlJication There are four classes of functional EMG data set [ 111 each of which containing 40 trials, half for training and half for testing. 24 features are extracted from these trials. Table I1 is the classification rates of four subjects. Slightly higher classification rate are acquired by the fuzzy classifier. In addition, as has been shown in [14], fuzzy classifier worked out more consistent outputs indicated by its smaller output variance than that of ANN classifier. IV. DISCUSSION A.  Selection of rule number The more rule number, the more partitions in the input space, and the more computations in inference and training. Within a certain limit, the performance of the system is improved as more rules are used because of more preciseness in matching the pattern distributions. However, with too many rules, the performance deteriorates as the system over-fits the training data. In order to identify the appropriate rule number, the system starts from small rule number which is usually triple the total number of classes. The rule number is then increased until the classification rate on test set does not improve any more. B. About the training In the fuzzy system, the hzzy seta are initialized by the cluster parameters from ISO-DATA, and the outputs in the conclusions are initialized by the “normalized vote” among the classes within the cluster. The initialization guarantees good start point for the rule base demonstrated by the fact that the M[SE and classification error rate are almost optimal even without training. After only a few epochs of training, the fuzzy system is expected to have more reliable generalization lbecause of the fine tune of fuzzy sets and rule conclusion. As to ANN, the weights have to be initialized randomly and the training process is usually much longer as a consequence. A typical relation of classification error rate vs. training epoch number is demonstrated in fig. 3. The data se1 is EMG signal of above- mentioned-subject No. 1. It can be found that FUZ works out good results (MSE and error rate) even without any training at all. In mathematical sensa, the triangular membership function is not differentiable at the three sharp points. However, hardly does it make any side effect in training. The training, by (9)-( 1 1) never generates illegal membership functions. C. Hardware implementation Fuzzification, inference and defuzificatioin of the system use only economical arithmetic computations. It can be implemented on general- purpose micro-controllers with low cost and fast running speed [ 131. Some fuzzy logic co-processors can also realize the system conveniently. V. CONCLUSION We have proposed a new fuzzy classifier architecture. The training data set is clustered by Basic ISODATA algorithm. Fuzzy sets are then defined based on these clusters with triangular membership function. Each cluster will yiield one rule. The outputs in the rule conclusion are initially determined by the “normalized vote” in the corresponding cluster. Fuzzy sets and conclusion outputs are adjusted by error back-propagation method afterwards. The proposed fuzzy system is simple in structure, and can easily be trained and implemented. Moreover, its classification performance is generally better than ANN. Linguistic rules can also be integrated to the rule base. VI. ACKNOWLEDGMENT This work is supported in part by The University of Hong Kong Research Grants. The authors want to thank Dr. B. Hudgins and Mr. C .  A, K. Englehart 48 1 at the Institute of Biomedical Engineering, University of New Brunswick (UNB), Canada, for providing the EMG data set. REFERENCES S. Abe, M.-S. Lan, “A method for fuzzy rules extraction directly from numerical data and its application to pattem classification”, IEEE Trans. on Fuzzy Systems, FEB 1995, VOL. 3, NO. 1, pp. S. Horikawa, T. Furuhashi, Y. Uchikawa, “On fuzzy modeling using fuzzy neural networks with the back-propagation algorithm”, IEEE Trans. on Neural Networks, SEP 1992, VOL. 3, NO. 5 ,  pp. N. Imasaki, J. Kiji, M. Arai, “Framework for fuzzy neural networks”, Science of ArtlJicial Neural Networks, 1992, Dennis W. Ruck, Editor, Proc. S. Mitra, S. K. Pal, “Fuzzy multi-layer perceptron, inference and rule generation”, IEEE Trans. on Neural Networks, JAN 1995, VOL. 6, NO. 1, pp. C.-T. Sun, “Rule-base structure identification in an adaptive-network-based fuzzy inference system”, IEEE Trans. on Fuzzy Systems, FEB 1994, VOL. 2, Y. J. Lee, V. Cherkassky, J. R. Slagle, “Adaptive fuzzy-rule-based classifier”, World Congress on Neural Networks, JUN 1994, San Diego, VOL. I, L.-X. Wang, J. M. Mendel, “Generating fuzzy rules by learning from examples”, IEEE Trans. SMC, R. Katayama, Y. Kajitani, Y. Nishida, “A self generating and tuning method for fuzzy modeling 18-28 80 1-806 SPIE 1710, pp. 535-546 51-63 NO. 1, pp. 64-73 pp. 699-704 NOVDEC 1992, VOL. 22, NO. 6, pp. 1414-1427 X d  using interior penalty method”, Proc. of the 2”d intemational conference on Fuzzy Logic and Neural Networks, JUL 1992, Japan, pp. 357- 360 B. Kosko, “Fuzzy systems as universe approximators”, IEEE Trans. on Computers, R. 0. Duda, P. R. Hart, “Pattern classification and Scene analysis”, John Wiley & Sons, USA, 1973 B. Hudgins, P. A. Parker, “A new strategy for multifunction myoelectric control”, IEEE Trans. BME, JAN 1993, VOL. 40, NO. 1, pp. R. P. Lippmann, “An introduction to computing with neural nets”, IEEE ASSP Magzine, APR 1987, VOL. 4, NO. 2, pp. 4-22 J. Yan, M. Ryan, J. Power, “Using fuzzy logic: towards intelligent systems”, Prentice Hall, New York, 1994 Y.-S. Yang, F. K. Lam, F. H. Y. Chan, Y.-T. Zhang, P. A. Parker, “A new fuzzy approach for pattem recognition with application to EMG classification”, Proceedings of International Conference on Neural Network (ICNN’96), Vol 11, pp 1109-1 114, June 1996, Washington D.C. Y.-S. Yang, “Fuzzy classification of biomedical signals”, Master of philosophy thesis, The University of Hong Kong, Oct. 1996, Hong Kong NOV 1994, VOL. 43, NO. 11, pp. 1329-1333 82-94 [I61 Y.-S. Yang, H. Nguyen, F. H. Y. Chan, F. K. Lam, “Fuzzy classification of biomedical signals”, Proceedings of International Conference on Biomedical Engieering (BME’96), pp 28-30, June 1996, Hong Kong Fig. 1 An exemplar fuzzy set definition via clusters in two-dimensional input space 482 Triangular Fuzzifiers 0.55 0.45 0.35 Fig. 2 Network representation of fuzzy classifier with triangular membership functions 0.22 0.25 5.5% 4.5% 0.30 0.38 11.5% 10.0% 0.34 0.43 12.5% 12.0% Table I. Comparison of ANN and Fuzzy classifier on synthetic overlapping data Subject Identification No. 1 No. 2 No. 3 No. 4 Classification Classification Rate by ANN Rate by FUZ 9 1 .3% 92.5% 78.8% 80.0% 96.3% 97.5% 83.8% 88.8% I I I I 0.25 1 0.42 I 0.47 I 19.5% i 20.0% tl Table II. Comparison of EMG recognition rates by ANN and Fuzzy classifier 483 . . . ... ... MSE - Error Rate 0.8 _______.  .MSE -Error Rate -! 0.8 .. 0.7 -~ 0.6 0.5 -- ; 0.5 0.4 0.4 -- '\\ 0 100 200 300 0 10 20 30 ANN training epoch FUZ training epoch .- -. .- Fig. 3. Mean square error over training set and classification error rate over test set vs. training epochs. (a) ANN system, training step size = 0.25; (b) FUZ system, training step size = 0.04, lsodata cluster number M = 13, rule = 13 x 4. 484 

A new fuzzy classifier with triangular membership functions

http://hub.hku.hk/bitstream/10722/46013/1/27271.pdf

A new fuzzy classifier with triangular membership functions

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub