The new fuzzy c-regression modeling (FcRM) are widely used in  order to fit switching regression models. Minimization of objective  function yields immediate estimates for different c regression  models. The functions of model, estimation technique and results  are discussed in this paper. A case study in miles per gallon (MPG)  of different cars using the FcRM modeling was carried out. The 3D  graph for significant independent variables for FcRM clustering is  shown in this study. The comparison between multiple linear  regression and FcRM modeling were done. The mean square error  (MSE) was used to find the better model. It was found that the  FcRM modeling with lower MSE to be the better model and has  great capability in predicting the dependent variable effectively

Adnan, Robiah

N. Nasibov, Efendi

Rusiman, Mohd. Saifullah

English

Universiti Teknologi Malaysia Institutional Repository

                           3D FCRM Modeling in Miles Per Gallon of Car                     35 4 3D FCRM MODELING IN MILES PER GALLON OF CAR  Mohd Saifullah Rusiman Robiah Adnan Efendi N. Nasibo     The new fuzzy c-regression modeling (FcRM) are widely used in order to fit switching regression models. Minimization of objective function yields immediate estimates for different c regression models. The functions of model, estimation technique and results are discussed in this paper. A case study in miles per gallon (MPG) of different cars using the FcRM modeling was carried out. The 3D graph for significant independent variables for FcRM clustering is shown in this study. The comparison between multiple linear regression and FcRM modeling were done. The mean square error (MSE) was used to find the better model. It was found that the FcRM modeling with lower MSE to be the better model and has great capability in predicting the dependent variable effectively. 4.1 OVERVIEW For the past few years, new fuzzy modeling has become popular because of their better explanation of describing complex systems. The modified version of FCM called the fuzzy c-regression model (FcRM) clustering algorithm [1] develops hyper-plane-shaped clusters. The FcRM assumes that the input–output  36                          Advances in Fundamental and Social Sciences                       data are drawn from c different regression models. This FcRM modeling is also known as a switching regression modeling. The minimization of the objective function in FcRM clustering simultaneously yields a fuzzy c-partitioning matrix of the data and the c regression models. This new technique has been widely used in engineering, science, medicine, economic and other fields. 4.2 DATA BACKGROUND A measure of fuel economy in automobiles or miles per gallon (MPG) used similarly in North America and the United Kingdom, although the Imperial gallon used in the UK is about 20% larger than the U.S. gallon. A metric term measures how many miles a vehicle can travel on one gallon of fuel. Most countries other than the US and UK use the SI (aka metric) units liter (0.22 Imperial gallon or 0.264 US liquid gallon) and km (0.621 statute miles). These can be combined to either km/L (efficiency) or L/100km (consumption). MPG figures are 20.095% higher in the UK than in the U.S. for the same real fuel economy. The research on MPG done by Larson et. al in [2] was about the city and highway MPG prediction models (a pilot study). In this research, in order to comply with federal law, automobile manufacturers must display window stickers on automobiles leaving their factories. This paper presents an inexpensive statistical method of predicting city and highway mpg estimates which could be possible alternative methods. Another research was done by Jun Meng and Xiangyin Liu in [3]. They use the data mining theory to construct a BP (back propagation) neural network model to predict MPG (mile per gallon). The six independent variables are number of cylinders, displacement, horsepower, weight, acceleration and model year.  In this paper, the data were gained fom SPSS Software version 10. The data were collected involving 400 cars. The dependent variable is miles per gallon (MPG), whereas the independent variables are engine displacement in cubic inches,                            3D FCRM Modeling in Miles Per Gallon of Car                     37 horsepower, weight of cars in lbs., time to accelerate from 0 to 60 mph, model year, country of origin (USA, European and Japanese), number of cylinders and cylinder filter.  The FCRM modeling proposed by Harthway and Bezdek in 1993 develops hyper-plane-shaped clusters. Kim et al. [4] successfully applied FcRM to construct fuzzy models in two phases algorithms which is c clusters are firstly identified through the FCRM cluster and a supervised learning algorithm further adjusts the obtained parameters to improve the modeling accuracy. The number of clusters (rules), c, is fixed and assigned by the user. In [5], for an unknown system, the appropriate number of clusters (rules) is supposed to be unknown and could be gained by using Equation (4.6). 4.3 METHODOLOGY Firstly, the analysis of influential and outlier data should be done to the data to discard unimportant data due to human error, machine error or environment error. The analysis used like Pearson standardized residual (outliers Y test) in [6] and [7], Leverage (outliers X test) and DFBETA (influential test) in [7]. However, there are no conditions needed in FcRM modeling.  A switching regression model is specified by         ci xfy iiii ≤≤+= 1       ,);( εθ                            The optimal estimate of θ  depends on assumptions made about the distribution of random vectors or iε . Generally, the iε are assumed to be independently generated from some pdf  p(ε; η, σ) such as the Gaussian distribution with mean 0 and unknown standard deviation iσ ,   222)(21),;( σηεπσσηε−−= ep                               38                          Advances in Fundamental and Social Sciences                        Based on the Hathaway and Bezdek algorithm in [1] and [8];  (1)   Fix the number of cluster c, 2 ≤ c. Choose the termination tolerance  δ > 0. Fix the weight, w,  w > 1 and initialise fcM∈(0)U  randomly. (2)   Estimate cθθ ,...,1  simultaneously  by modifying the fuzzy c-means algorithm (FCM). If the regression functions );( ii xf θ are linear in the parameters ,iθ  the parameters can be obtained as a solution of the weighted least squares:    [iθ =T -1 Tb i b b iX WX ] X WY                                  (3)   Calculate the objective function:   21 1||);(||.}]{,U[ ijijcidjwijiw xfyuE θθ −=∑∑= =          (4) Make iterations  in  order  to minimize the objective function. Repeat for l = 1, 2, . . . until δ  ||UU|| )1()( <− −ll . The steps are described as follows:  Step 1 :  Calculate model parameters )(liθ to                 globally minimize (4). Step 2 :  Update  U   with ][ )1( −= liijij EE θ ,  to               satisfy:                     ∑=−⎟⎟⎠⎞⎜⎜⎝⎛=ckwkjijlijEEu112)( 1    for  0>ijE                          and 1 ≤ i ≤ c                      otherwise 0=iju  if ,0>ijE  and                            3D FCRM Modeling in Miles Per Gallon of Car                     39         ∈iju  [0, 1] with ( cjj uu ++ ...1 ) = 1          until δ  ||UU|| )1()( <− −ll                          In the FCRM clustering algorithm, the number of clusters, c, is fixed and assigned by the user. In practice, the appropriate number of clusters is usually decided with the aid of the cluster validity criterion like the Bezdek’s partition coefficient [9] as follows,                  )(1 12NVNhciihPC∑∑= ==μ.                                          The optimal number c is chosen when PCV  is closest to 1.  Takagi and Sugeno [10,11] has introduced a fuzzy rule-based model and also called as an affine T-S fuzzy model [12], described as follows:        THEN          is      andisIF:01111ininiiinniiaxaxay  A x  A  x R+++= ??                       where iR  denotes the ith IF-THEN rule             i = 1, 2, …, c where c is the number of rules           ,,...,1  , nmx m =  are individual input variables  imA  are individual antecedent fuzzy sets  nkaik ,...,1  , =  are consequent parameters  ia0  denotes a constant  ℜ∈iy  is the output of each rule  The output of the fuzzy model yˆ  is inferred in [13,14] if the singleton fuzzifier, the product fuzzy inference and the centre average defuzzifier are applied:  40                          Advances in Fundamental and Social Sciences                                 11( )  ˆ( )ci iiciiw yyw===∑∑xx                                            where   ∏==××=nmmimniniii xAxAxAxAw12211 )()(...)()()(x      denotes the degree of fulfillment of the antecedent, that is, the level of firing of the ith rule.   The consequent parameters can be found directly from the FcRM program output. The antecedent fuzzy sets imA  are achieved by projecting the membership degrees in the fuzzy partitions matrix U  onto the axes of individual antecedent variable mx  and then to approximate it by a normal bell-shaped membership function. Hence, each antecedent fuzzy set imA  is calculated from the sampled input data T1 ],...,[ nhhh xxx =  and the fuzzy partition matrix U [ ]igμ=  as follows [15,16] :   21( ) exp2iqiq iqzA zμσ⎧ ⎫⎛ ⎞−⎪ ⎪= − ⎜ ⎟⎨ ⎬⎜ ⎟⎪ ⎪⎝ ⎠⎩ ⎭,                              with the mean and standard deviation are respectively given as   21 11 1( )   andN Niih qh ih qh qi ih hq qN Nih ihh hx xμ μ αμ σμ μ= == =−= =∑ ∑∑ ∑.                             3D FCRM Modeling in Miles Per Gallon of Car                     41 4.4 NUMERICAL EXAMPLES Firstly, the analysis of influential and outlier data should be done to the data. The analysis used are Pearson standardized residual (outliers of Y), Leverage (outliers of X) and DFBETA (influential test). Nine data are discarded due to missing value of MPG. From the analysis of influential and outlier data, it was found that seven set of data should be discarded.  The dependent variable is miles per gallon (MPG), whereas the significant independent variables are endisplacement ( 1x ), horsepower ( 2x ), weight ( 3x ), year ( 4x ), origin ( 5x ), cylinderno ( 6x ), cylinderflt ( 7x ) and accellerationt ( 8x ). Figure 4.1 shows the individual scatter plots for MPG versus 1x  to 8x . The plots indicate the negative relationship between MPG versus endisplacement, horsepower, weight and cyclinder number. The positive relationship is shown between MPG versus cylinderflt whereas the scatter plot for MPG versus year, origin and accellerationt indicate no any relationship.  42                          Advances in Fundamental and Social Sciences                        Figure 4.1. Scatter plot for MPG versus independent variables.  The FcRM clustering for the data were analysed by using Matlab software. Table 4.1 shows the optimal value for the number of clusters is two for 1x  to 8x  since the value PCV  is close to 1 when c = 2.   Table 4.1. The value of PCV  for Y versus 1x  to 8x . Cluster Number (c) 2 3 4 5 Y vs 1x  0.898 0.881 0.885 0.868 Y vs 2x  0.915 0.887 0.877 0.887 Y vs 3x   0.914 0.908 0.902 0.889 Y vs 4x  0.892 0.872 0.889 0.883 Y vs 5x  0.937 0.912 0.897 0.909 Y vs 6x  0.937 0.906 0.906 0.908 Y vs 7x  0.924 0.916 0.906 0.912 Y vs 8x  0.936 0.920 0.912 0.905                            3D FCRM Modeling in Miles Per Gallon of Car                     43  Graphs in Figure 4.2 indicate the individual cluster plot for 1x  to 8x .     50 100 150 200 250 300 350 400 450 50005101520253035404540 60 80 100 120 140 160 180 200 220 240-50510152025303540451500 2000 2500 3000 3500 4000 4500 5000 55005101520253035404570 72 74 76 78 80 821015202530354045  Y vs 1X   Y vs 2X   Y vs 3X  1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 310152025303540453 3.5 4 4.5 5 5.5 6 6.5 7 7.5 810152025303540451 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 210152025303540458 10 12 14 16 18 20 22 24 261015202530354045  Y vs 5X   Y vs 6X   Y vs 7XFigure 4.2. Individual cluster plot.    44                          Advances in Fundamental and Social Sciences                       The memberships for antecedent parameters are calculated using formula (4.10). Hence, only two parameters are significant which are origin and cyclinderflt with the smallest MSE. The number of clusters chosen is two since its PCV  is the closest to 1 as summarized in Table 4.2.     Table 4.2. The value of PCV  for Y versus 5x and 7x . Number of clusters, c 2 3 4 5 PCV  0.935 0.917 0.908 0.919    A significant affine T-S fuzzy model described as follows:    627.4411.6578.2  THEN            is andisIF:7511721511++= xxy      A x  A  x R    187.4455.11359.2  THEN            is andisIF:7522722512++= xxy   A x  A  x R   where the antecedent parameter is described in detail in Table 4.3.                                             3D FCRM Modeling in Miles Per Gallon of Car                     45    Table 4.3. Details of the antecedent parameter . iR  i = 1 i = 2 11 in x Ai     : μ                          : σ 215.210 109.09 168.169 90.629 22 in x Ai    : μ                : σ 114.871 41.777 94.938 31.173 33 in x Ai    : μ                : σ 2939.726 717.119 3047.907 991.107 44 in x Ai    : μ                : σ 75.753 3.475 76.142 3.812 55 in x Ai    : μ                : σ 1.399 0.709 1.81 0.864 66 in x Ai    : μ                : σ 5.628 1.658 5.248 1.746 77 in x Ai    : μ                : σ 1.701 0.458 1.782 0.413 88 in x Ai    : μ                : σ 15.268 2.926 16.154 2.216   46                          Advances in Fundamental and Social Sciences                       0 50 100 150 200 250 300 350 40000.10.20.30.40.50.60.70.80.91 Figure 4.3. Membership function plot for y   versus 5x and 7x .     Figure 4.3 represents the membership function graph for MPG versus 5x and 7x  with the optimal two clusters. Figure 4.4 shows the three-dimension graph of clustering for MPG versus 5x and 7x  with two clusters. The middle plane is the multiple linear regression plane for all data. The other two planes are the two clustering planes with two different multiple linear regression planes.                             3D FCRM Modeling in Miles Per Gallon of Car                     47 1 1.52 2.5311.521015202530354045 Figure 4.4. Three-dimension FcRM clustering graph for y versus 5x and 7x .    In finding the better model, the mean square error (MSE) is used as follow:  1. For Multiple Linear Regression Model ( )2ˆ1 ∑ −−= ii YYpNMSE                    2. For FcRM Model ( )2ˆ1 ∑ −= ii YYNMSE                    where iY  denotes the real data,                Yˆ  represents the predicted value of iY ,                N is the number of data, and                p is the number of parameters.   48                          Advances in Fundamental and Social Sciences                       The comparisons between these two model are summarized in Table 4.4.    Table 4.4. The comparisons between 2 models.  Model Variable Chosen MSE Multiple Linear Regression Model   , , , , 4321 XXXX  765  , , XXX  (Significant Variable) 8.246FcRM Model  , , , , 4321 XXXX    , , , 8765 XXXX  (All Variables) 8.932  75   and XX  (Significant Variable) 7.8484.5 CONCLUSION A new model called FcRM has been proposed in analyzing a continuous data where no assumptions are needed in the analysis. The minimization of objective function yields immediate estimates for different c regression models. The comparison modeling between FcRM and multiple linear regression modeling indicate that FcRM modeling appeared to be the better model with the lower MSE. This FcRM modeling could be proposed as one of the best model in analyzing mainly in a complex system. Hence, the value of MPG could be predicted based on the variable of origin and cylinderflt. The best cars that could increase MPG come from Japan, followed by Europe and America. Cars with cylinder filter have better MPG than cars without cylinder filter.                            3D FCRM Modeling in Miles Per Gallon of Car                     49 4.6 REFERENCES [1] Hathaway, R. J., and  Bezdek,  J. C.: ‘Switching regression models and fuzzy clustering’, IEEE Trans. Fuzzy Syst., 1993, 1, (3), pp. 195–204. [2] Larson, E. C., Foster, K. A., Hopfe, M. W. : ‘City and highway MPG prediction models (a pilot study)’, Proceedings Decision Sciences Institute 1994 Annual Meeting, Part vol.2, 1994, pp. 1316-17. [3] Jun Meng and Xiangyin Liu : ‘MPG Prediction based on BP Neural Network’, IEEE on Industrial Electronics and Applications, International Conference, 2006, pp. 1-3. [4] Kim, E.,  Park, M., Ji, S., and  Park, M.: ‘A  new approach to fuzzy modeling’, IEEE Trans. Fuzzy Syst., 1997, 5, (3), pp. 328–337. [5] Chuang, C.C., Su, S.F., and Chen, S.S.: ‘Robust TSK fuzzy modeling for function approximation with outliers’, IEEE Trans. Fuzzy Syst., 2001, 9, (6), pp. 810–821. [6] Agresti,  A. :  ‘An  Introduction  to Categorical Data Analysis’ ,John Wiley & Sons, Inc, New York,  1996. [7] Norusis, M. J. : ‘SPSS for Windows Advanced Statistics Release 6.0’ , SPSS Inc, USA, 1993. [8] Abonyi, J.  and  Feil, B. : ‘Cluster  analysis  for  data mining and system identification’, Springer, USA, 2007. [9] Bezdek, J.C. : ‘Cluster  validity with  fuzzy  set ’, J. Cybern., 1974, 3, pp. 58–72. [10] Takagi, T., and Sugeno, M.: ‘Fuzzy identification of systems and its applications to modeling and control’, IEEE Trans. Syst., Man Cybernatic, 1985, 15, pp. 116–132. [11] Sugeno, M., and Kang, G.T.: ‘Structure identification of fuzzy model’, Fuzzy Sets    Syst., 1988, 28, (1), pp. 15–33. [12] Babuska, R.: ‘Fuzzy modeling for control’, Kluwer Academic Publishers, Boston, 1988. [13] Wang, L.X. : ‘Adaptive  fuzzy  systems  and control: design and stability analysis’, Prentice-Hall, NJ, 1994. 50                          Advances in Fundamental and Social Sciences                       [14] Wang, L.X.: ‘A course in fuzzy systems and control’, Prentice-Hall, NJ, 1997. [15] Hoppner, F., Klawonn, F., Kruse, R., and Runkler, T.: ‘Fuzzy cluster analysis, methods for classification, data analysis and image recognition’, John Wiley & Sons, NJ, 1999. [16] Kim, E., Park, M., Ji, S., and Park, M.: ‘A new approach to fuzzy modeling’, IEEE Trans. Fuzzy Syst., 1997, 5, (3), pp. 328–337.  

3D FcRM modelling in miles per gallon of cars

http://eprints.utm.my/14283/1/RobiahAdnan2008_3DFcRMModellinginMilesPerGallon.pdf

3D FcRM modelling in miles per gallon of cars

Abstract

Similar works

Full text

Available Versions

Universiti Teknologi Malaysia Institutional Repository