Currently, it is recognized that manufacturing systems are complex in their structure and dynamics. Management, control and forecasting of such systems are very difficult tasks due to complexity. Numerous variables and signals vary in time with different patterns so that decision makers must be able to predict the behavior of the system.This is a necessary capability in order to keep the system under a safe operation. This also helps to prevent  emergencies and the occurrence of critical events that may put in danger human beings and capital resources, such as expensive equipment and valuable production. When dealing with chaotic systems, the management, control, and forecasting are very difficult tasks. In this article an application of neural networks and vector support machines for the forecasting of the time varying average number of parts in a waiting line of a manufacturing system having a chaotic behavior, is presented. The best results were obtained with least square support vector machines and for the neural networks case, the best forecasts, are those with models employing the invariants characterizing the system’s dynamics

Alfaro, Miguel D.

Sepúlveda, Juan M.

Ulloa, Jasmín A.

Currently, it is recognized that manufacturing systems are complex in their structureÂ and dynamics. Management, control and forecasting of such systems are very difficultÂ tasks due to complexity. Numerous variables and signals vary in time with differentÂ patterns so that decision makers must be able to predict the behavior of the system.This is a necessary capability in order to keep the system under a safe operation.Â This also helps to prevent Â emergencies and the occurrence of critical events that mayÂ put in danger human beings and capital resources, such as expensive equipment andÂ valuable production. When dealing with chaotic systems, the management, control,Â and forecasting are very difficult tasks. In this article an application of neural networksÂ and vector support machines for the forecasting of the time varying average numberÂ of parts in a waiting line of a manufacturing system having a chaotic behavior, isÂ presented. The best results were obtained with least square support vector machinesÂ and for the neural networks case, the best forecasts, are those with models employingÂ the invariants characterizing the system’s dynamics

Ulloa, Jasmí­n A.

Agora University Editing House: Journals

INT J COMPUT COMMUN, ISSN 1841-98368(1):8-17, February, 2013.Forecasting Chaotic Series in Manufacturing Systems by VectorSupport Machine Regression and Neural NetworksM.D. Alfaro, J.M. Sepúlveda, J.A. UlloaMiguel D. Alfaro, Juan M. SepúlvedaJasmín A. UlloaDepartment of Industrial Engineering, University of Santiago of Chile3769 Ecuador Ave. Santiago, Chile.E-mail: miguel.alfaro@usach.cl, juan.sepulveda @usach.cljasmin.ulloa@usach.clAbstract:Currently, it is recognized that manufacturing systems are complex in their structureand dynamics. Management, control and forecasting of such systems are very difficulttasks due to complexity. Numerous variables and signals vary in time with differentpatterns so that decision makers must be able to predict the behavior of the system.This is a necessary capability in order to keep the system under a safe operation.This also helps to prevent emergencies and the occurrence of critical events that mayput in danger human beings and capital resources, such as expensive equipment andvaluable production. When dealing with chaotic systems, the management, control,and forecasting are very difficult tasks. In this article an application of neural networksand vector support machines for the forecasting of the time varying average numberof parts in a waiting line of a manufacturing system having a chaotic behavior, ispresented. The best results were obtained with least square support vector machinesand for the neural networks case, the best forecasts, are those with models employingthe invariants characterizing the system’s dynamics.Keywords: chaos; forecast; neural networks; vector support machines; manufactur-ing systems1 IntroductionManufacturing systems are conceived as complex ones; although the complex term does nothave a unique deﬁnition [1] it is possible to distinguish two kinds of complexities in productionsystems: a) structural complexity or static complexity dealing with the number of system’scomponents and their relationships, and b) dynamic complexity dealing with the uncertaintyin the systems behavior [2]. It may seem paradoxical that an artiﬁcial system engineered formaking a set of given tasks had its own laws as if it was a natural system. This is due to thefact that production systems are becoming everyday more complex by the technological progressand the transformation of the supply chain. Flexible manufacturing machinery, global markets,and supply network relationships are typical examples of such changes. Managing these systemsto bring them under control is today a diﬃcult task. The dynamic complexity of productionsystems has been demonstrated by the kinds of behavior that they can exhibit, among these achaotic behavior [3], [4], [5]. Several metrics have been proposed for measuring the complexity ofmanufacturing systems [2], [6], [7], [8]. These studies relate metrics of performance with metricsof complexity. In this article, a way of control by forecasting the system’s behavior is shown.For this purpose, a time series of the average number of parts in the waiting line of a chaoticmanufacturing system is utilized [3]. As forecasting methods, Support Vector Machines (SVMs)and Artiﬁcial Neural Networks (ANNs) have been selected because these methods can distinguishchaotic patterns and therefore they can predict the evolution of an observed control variable.In [9] SVMs have been used for support vector regression (SVR) analysis of several exchangerates with respect to the US dollar. In the present work, a least square support vector regressionCopyright c© 2006-2013 by CCC PublicationsForecasting Chaotic Series in Manufacturing Systems by Vector Support Machine Regressionand Neural Networks 9(LS-SVR) with less computational eﬀort than the one reported in [10] is proposed. In [11] anANN model for forecasting the observed error of monitoring units of the sea level in Singapore ispresented. In [12] an ANN is constructed for forecasting the behavior of a diode having a chaoticpattern; in this case the local dimension and the time delay are proposed for determining thenetwork architecture.The originality of this work consists of the study of the performance of two forecastingtechniques: LS-SVM and ANN as applied to chaotic series. Similar studies have been made butfor series that are not chaotic [13], [14]. Also, it is a novel application in the manufacturing area.The paper is organized as follows: section 2 shows in a summarized way the manufacturingsystem from which the time series was obtained; section 3 presents the methods utilized forthe forecasting; section 4 shows the results of the analysis of the time series by using non-linear dynamic systems (NLDS) theory. Forecasting results are detailed in Section 5. Finally,conclusions and research directions are given.2 System under study2.1 Variable to be analyzedThe system under study is described in [3]. The variable to be analyzed is the average in timeof the number of parts in the waiting line of a ﬂexible manufacturing system. By utilizing a timeseries of this variable the system’s dynamics is characterized by means of the theory of non-lineardynamic systems. The machining shop is formed by three diﬀerent machines producing threetypes of parts. Each part has a set of operations which can be executed in diﬀerent machinesaccording to the operations sequence. Figure 1 shows the layout of the system under study.Figure 1: Machining shop layout.2.2 Operation of the system• The arrivals rate and service rate are such that the system is under equilibrium; that is,the number of parts neither tends to zero nor inﬁnity.• Upon arrival of a part, a function f assigns the part to a machine able to perform theoperation and that has the least number of parts in queue.• The priority of the queue of each part type at any machine is ﬁrst-in ﬁrst-out (FIFO).10 M.D. Alfaro, J.M. Sepúlveda, J.A. Ulloa• The machine works cyclically with a kind of part, during a time interval equivalent to thetime needed to complete the stock of that kind of part at the k-th machine. The function gkmanages the execution cycle according to the part type. The values of plant parameters areshown in Table 1, where βj is the arrival rate of the j-th part type (number of parts/timeunit) and Oij is the i-th operation of the j-th part. The values in the table represent theoperation time at each machine.3 Forecasting Methods UtilizedResearch on forecasting models has received considerable attention over the last 50 years.Currently, there exist numerous forecasting methods [15]. For the case of chaotic systems, thevery theory on NLDS provides forecasting methods [16]. In this article, NLDS theory is usedas a base for constructing ANN and VSM models. These techniques are proposed due to theircapacity for recognizing chaotic patterns.3.1 Artificial Neural NetworksAn artiﬁcial neural network (ANN) is a computational model of the brain. It consists of alimited number of connected elements (neurons) and it is distributed in an input layer, one ormore hidden layers, and an output layer.An ANN is a mathematical structure that allows patternrecognition; in this work we use a Back-Propagation type of network, as shown in ﬁgure 2.As it is known that the system is chaotic, it is proposed as the number of neurons in theinput layer the dimension of the phase space. This value is obtained by the method of falseneighbors. The temporal distance between the input variables corresponds to the time delayin the construction of the systemďż˝s attractor. This value is obtained from the average of themutual information [16]. The transfer function is of the sigmoidal type and the network is deﬁnedby the equation (1).xt = β0 +n∑i=1βif(sωi0 +d∑j=1ωijxt−j) (1)Where n is the number of neurons in the hidden layer, d is the number of neurons in theForecasting Chaotic Series in Manufacturing Systems by Vector Support Machine Regressionand Neural Networks 11Figure 2: Neural Network for Forecastinginput layer, s is the standard deviation of the weights matrix and βi, ωij are the weights. Thenumber of neurons in the hidden layer is due to the following empirical relationship 2:Nobs10≥ (Ne + 1)Nc + (Nc + 1)Ns (2)Where Nobs is the number of observations, Ne, Nc and Ns are the number of neurons in theinput layer, hidden layer, and output layer, respectively.3.2 Support vector machines for least squares regressionSupport vector machines (SVM) algorithms emerged from the artiﬁcial intelligence ﬁeld andthey have been successfully used in a variety of applications for problems of classiﬁcation andregression. The least-squares vector support machine is a modiﬁed version of a standard VSM forregression; the model is trained by solving a linear system instead of a quadratic programmingoptimization model [10]. The LS-SVM are closely related with regularization networks andGaussian processes, buy they emphasize and exploit its interpretation from the viewpoint of theoptimization theory. The general formulation for a LS-SVR is shown in (3):y = wT f(x) + b (3)Where x is the input vector of the data series, Y is the output vector. Parameters w and bare obtained from the optimization problem given by equations (4), (5).Min τ(w, b, e) =12‖w‖2 + γ12N∑i=1e2i (4)s.t. (yi − (〈w, xi〉) + b) = ei ∀i = 1, ...N (5)In (4), γ is an arbitrarily chosen parameter. For learning, the kernel utilized is the radialbase function (RBF) (6).k(xi, xj) = exp(−‖yi − xi‖2σ2) (6)12 M.D. Alfaro, J.M. Sepúlveda, J.A. UlloaThe architecture of the LS-SVR [17] is shown in ﬁgure 3.Figure 3: Architecture generated by regression SVM4 Characterization of the system’s dynamicsIt has been stated that the system has a chaotic behavior; in ﬁgures 4, 5, 6, 7, 8, 9 the analysisconﬁrming this assumption is shown. In ﬁgure 4 the time series plot is seen, ﬁgure 5 shows theFourier spectrum where the erratic nature of the series is veriﬁed; ﬁgure 6 presents the timedelay corresponding to the ﬁrst minimum of the mutual information average. Figure 7 shows thephase space dimension d = 5, which has been obtained by the percentage of false neighbors [16].However, ﬁgure 8 shows that the local dimension is 3 (correlation dimension 2.784). Finally, inﬁgure 9 is observed that the highest Lyapunov’s exponents value is 0.41, which corroborates thesignal’s chaotic character.Figure 4: Average of the number of parts intimeFigure 5: Fourier power spectrumForecasting Chaotic Series in Manufacturing Systems by Vector Support Machine Regressionand Neural Networks 13Figure 6: Mutual information average Figure 7: Dimension of phase spaceFigure 8: Correlation Dimension Figure 9: Lyapunov’s exponents5 Forecasting models resultsIn order to measure the performance of the models, the Apropiability Index (IA) and theNormalized root of quadratic error (RMS) are used, as deﬁned by (7) and (8) respectively:IA = 1−∑ni=1(yi − y,i)2∑ni=1(|yi|+ |y,i|)2(7)RMS =√∑ni=1(yi − y,i)2∑ni=1(yi)2(8)Where, yi is the value of the average number of parts at period i, y,i is the value predictedby the model, and n is the number of forecasted periods. The IA indicates the proportion ofthe variance that is explained by the model; values greater than 0.9 are expected. Whereas, theRMS compares the error between the desired output and the one generated by the model; valuesclose or below to 0.1 are expected.The series has 12001 data or periods, which correspond to the time persistent average numberof parts in a time interval of 0.15 time units. A 70% is used for training and the remaining 30%for validation.5.1 Experimental Results for ANNFor the forecasting model, a neural network is designed based on the time delay τ and thedimension d which were obtained from the chaotic system’s characterization. Five forecastingmodels are constructed with 1, 3, 5, 7 and 10 neurons in the input layer, with a time delay τ = 4in each model. The number of neurons in the hidden layer is obtained by the relationship (2) for14 M.D. Alfaro, J.M. Sepúlveda, J.A. Ulloaeach model; thus it is obtained: 279, 167, 119, 93, 69 neurons, respectively. All of the modelswere implemented by using the MATLABTMANN toolbox. Figures 10, 11, and 12 show theresults for the cases of 3, 5 and 10 neurons in the input layer.Figure 10: Forecast with three neurons inthe input layerFigure 11: Forecast with ﬁve neurons in theinput layerFigure 12: Forecast with ten neurons in the input layerTable 2 show the values of IA and RMS indicators for the diﬀerent models.It is observed that the model of 10 neurons is superior in 0.03% to the model with ﬁve neuronsin the input layer. However, if the RMS of the same models are compared, the model with 10neurons is greater in 3.65%. In a similar analysis between the models with three and ten neurons,it can be seen that the diﬀerence of the IAs is 0.05% and the RMS’s 0.67%. Then, it is possibleto state that the best models can be found between the ones with three and ﬁve neurons in theinput layer. These values are exactly those corresponding to the local dimension and the globaldimension in the phase space.5.2 Experimental Results for LS-SVRThe values of the RBF kernel parameters associated with the optimization problem describedby equations (4), (5) and (6) are γ = 256 and σ = 8. As time delay in the input variables τ =4 has been used. Likewise the ANN case, ﬁve models have been constructed with 1, 3, 5, 7 and10 input vectors. The models were implemented by using MATLABTM LSSVMlab1.7. Figures13, 14 and 15 show results for 3, 5 and 10 input vectors.Forecasting Chaotic Series in Manufacturing Systems by Vector Support Machine Regressionand Neural Networks 15Figure 13: Forecast with three input vectors Figure 14: Forecast with ﬁve input vectorsFigure 15: Forecast with ten input vectorsTable 3 shows the values of the indicators IA and RMS for the diﬀerent models.The results in Table 3 are not the expected ones; the best results correspond to seven and teninput vectors but not between dimensions three and ﬁve as with ANNs. The same experimentwas executed without time delay, that is τ = 1 was assumed. The results are shown in Table 4.Again, it is observed that best results are for seven and ten input vectors. Thus, for thisspeciﬁc case, it is veriﬁed that does not exists a behavior pattern based on NLDS characterization.5.3 Comparison between experimental results for ANN and LS-SVREven though the diﬀerence between the IA of the best ANN and LS-SVR models is 0.11%in these models (ﬁve neuron for ANN and seven input vector for LS-SVR) the LS- SVR’s RMSvalue is practically a half the ANN’s. Hence, it is possible to conclude that the best model is theone constructed with LS-SVR. Nevertheless, it is observed that in general terms both approachesperform adequately.16 M.D. Alfaro, J.M. Sepúlveda, J.A. Ulloa6 ConclusionsAs seen, the best results were obtained with least square support vector machines. Similarresults have been reported for SVR and ANN for non-chaotic series [13], [14]. Notwithstanding,it can be stated that both models are eﬃcient for the forecasting of a chaotic series obtained froma ﬂexible manufacturing system. According to the results, it can be concluded that the system’sbehavior can be predicted in one time step, that is 0.15 time units. As research directions, itis suggested to develop models able to predict a longer time interval. For this purpose, ongoingwork by the authors is addressing the use of the inverse of the Lyapunov’s exponent in order todetermine the number of neurons in the output layer for the ANN case, and of the number ofoutput vectors for the LS-SVR.AcknowledgmentsThis research has been supported by DICYT ( Scientic and Technological Research Bureau)of The University of Santiago of Chile (USACH) and Department of Industrial Engineering.Bibliography[1] EIMaraghy H., Kuzgunkaya 0., Urbanic R., Manufacturing Systems Configuration Complex-ity, CIRP ANNALS-MANUFACTURING TECHNOLOGY, ISSN 0007-8506, 54(1): 445-450,2005.[2] Papakostas N., Efthymiou K., Mourtzis D., Chryssolouris G., Modelling the complex-ity of manufacturing systems using nonlinear dynamics approaches, CIRP ANNALS-MANUFACTURING TECHNOLOGY, ISSN 0007-8506, 58: 437-440, 2009.[3] Alfaro M., Sepúlveda J. Chaotic behavior in manufacturing systems, INT. J. PRODUCTIONECONOMICS, ISSN 0925-5273, 101: 150-158, 2006.[4] Papakostas N., Mourtzis D., An Approach for Adaptability Modeling in Manufacturing- Anal-ysis Using Chaotic Dynamics, CIRP ANNALS-MANUFACTURING TECHNOLOGY, ISSN0007-8506, 56(1): 491-494, 2007.[5] Donnera R., Scholz-Reiter B., Hinrichs U., Nonlinear characterization of the performanceof production and logistics networks, JOURNAL OF MANUFACTURING SYSTEMS ISSN:0278-6125, 27(2): 84-99, 2008.[6] Wu Y., Frizelle G., Efstathiou J., A study on the cost of operational complexity in customer-supplier systems, INT. J. PRODUCTION ECONOMICS, ISSN 0925-5273, 106(1): 217-229,2007.[7] Phukan A., Kalava M., Prabhu V.,Complexity metrics for manufacturing control architecturesbased on software and information flow, Computers and Industrial Engineering ISSN: 0360-8352, 49(1): 1-20, 2005.[8] Wang H., Hu S., Manufacturing complexity in assembly systems with hybrid configurationsand its impact on throughput, CIRP ANNALS-MANUFACTURING TECHNOLOGY, ISSN0007-8506, 59: 53-56, 2010.Forecasting Chaotic Series in Manufacturing Systems by Vector Support Machine Regressionand Neural Networks 17[9] Huang S., Chuang P., Wub C., Lai H., Chaos-based support vector regressions for exchangerate forecasting, EXPERT SYSTEMS WITH APPLICATIONS, ISSN: 0957-4174, 37: 8590-8598, 2010.[10] He K., Lai K., Yen J., A hybrid slantlet denoising least squares support vector regressionmodel for exchange rate prediction, PROCEDIA COMPUTER SCIENCE ISSN 1877-0509,1: 2397-2405, 2010.[11] Sun Y., Babovic V., Chan E., Multi-step-ahead model error prediction using time-delayneural networks combined with chaos theory, JOURNAL OF HYDROLOGY, ISSN 0022-1694, 395(1): 109-116, 2010.[12] Hanias M., Karras D. On ef?cient multistep non-linear time series prediction in chaotic dioderesonator circuits by optimizing the combination of non-linear time series analysis and neu-ral networks, ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE ISSN0952-1976, 22(1): 32-39, 2009.[13] Vanajakshi L., Rilett L.,A Comparison of The Performance Of Artificial. Neural Networksand Support Vector Machines for the Prediction of Traffic Speed, IEEE INTELLIGENT VE-HICLES SYMPOSIUM ISBN 0-7803-8310-9, 194-199, Parma, Italy June 2004.[14] Yoon H.,Jun S, ,Hyun Y.,Bae G., Lee K.,A comparative study of artificial neural networksand support vector machines for predicting groundwater levels in a coastal aquifer, JOURNALOF HYDROLOGY, ISSN 0022-1694, 396(1): 128-138, 2011.[15] Scott J., it Principles of Forecasting, University of Pennsylvania, Kluwer Academic Pub-lisher 2001.[16] Abarbanel H., Analysis of observed chaotic data. New York, Springer-Verlag, 1996.[17] Vapnik V., Statistical Learning Theory, Wiley. Springer, New York. USA. 1998.

Forecasting Chaotic Series in Manufacturing Systems by Vector Support Machine Regression and Neural Networks

http://univagora.ro/jour/index.php/ijccc/article/view/163/pdf_2

Forecasting Chaotic Series in Manufacturing Systems by Vector Support Machine Regression and Neural Networks

Abstract

Similar works

Full text

Available Versions

Agora University Editing House: Journals