This paper discusses on-line modelling and forecasting

of carbon monoxide (CO) concentrations using Hybrid

Multilayered Perceptron (HMLP) Network. Th¢ HMLP

network is trained using Modified Recursive Prediction

Error (MRPE) algorithm

Ali, A.N.

Mashor, M.Y.

Raji, Usharani

Sadullah, A.F.

English

Repository@USM

Proceedings of the Intefnational Conference onRobotics, Vision, Information and Signal Processing ROVISP2005On-Line Modelling And Forecasting Of Carbon Monoxide Concentrations Using~- .. . ..~... .. ~nybrid Multilayered Perceptron-Network-- -- .Usharani Raji!, M.Y.Masho~,A.N.Ale, A.F. Sadullah4l,2.3Control and Electronics Intelligent System (CELIS) Research Group, ~chool ofE & Ec Engi,~eeri'lgUniversity Sains Malaysia, Engineering Campus,14300 Nibong Tebal, Seberang Perai Selatan.,Pulau Pinang,MalaysiaTel:+604-5937788 ext.6093, Email:vsharani@yahoo.comJ, yuso!@eng.usm.m/, nazri@eng.usm.m/4Engineering Innovation & Technology Development Unit,, University Sains Malaysia, Engineering Campus,14300Nibong Tebal, Seberang Perai Selatan,Pulau Pinang,MalaysiaTel:+604-5937788 ext.5050, Email:cefrhn@eng.usm.myAbstractThis paper discusses on-line modelling and forecastingof carbon monoxide (CO) concentrations using HybridMultilayered Perceptron (HMLP) Network. Th¢ HMLPnetwork is trained using Modified Recursive PredictionError (MRPE)· algorithm.' In the· literature, COconcentrations forecasting are always conducted usingoff-line modelling. In this preliminary study, on-linemodelling is introduced, which means networkparameters will be updated for each .data sample.Historical CO concentration values are used to provideon-line forecasting. The proposed on-line modelling willbe compared to off-line modelling. The performances of .those modelling are evaluated using index of coefficient(R2) and mean square erroE (MSE). On-line model arefound to perform better compared to off-line model.K,eywords:on-line, real. time, off-line" carbon monoxide,. forecasting, hybrid multilayered perceptronIntroduction. Carbon' Monoxide (CO) is produced from incomplete. burning of carbon containing fuels. CO is a colourless;odourless, tasteless and a very poisonous gas. Accordingto the Journal of American Medical Association(JAMA), 1500 people die annually due to accidentalcarbon monoxide poisoning and 10000 people seekmedical attention [1]. Motor vehicl~s are a major sourceofCo in urban areas.A lot of researches have been carried out using differentmethodology on CO concentrations forecasting. One ofthe methods was by using univariate linear stochasticmodels based on Box~Jenkins modelling. technique [2].This model sufficiently needs long historical data set formodel formulation. Another approach was by using BoxJenkins transfer function noise model (TFN), the752forecast performance were better compared to the firstapproach [3]: Besides that, Gaussian and regressionmodels were also implemented. for CO forecasting[4][5]. Another method for forecasting isimplementation of Neural Network (NN). NN have beenproved mathematically of representing non-linearsystems. A NN known as "Brainmaker" using backpropagation algorithm were used to Rredict COconcentration with an accuracy of R =0.69 [6].Forecasting on other gases using NN were reviewedsince not much of studies have been done specifically onCO. In another study, prediction of PM2.5 concentrationswas carried mit using multilayer neural.network. linearregression ,and persistence. The predictions produced bythose methods were compared and NN was found togive the best result [7]. In another work, an improvedneural network model, that combines principalcomponent'study and radial basis function (PCNRBF)were used to forecast hourly time series of RSP, NOxand NOi. The improved model was found to performbetter than simple RBF [8].In·the present study, CO concentrations forecasting willbe performed using Hybrid Multilayered PerceptronNetwork (HMLP) with Modified Recursive PredictionError Algorithm (MRPE). The on-line and off-linemodelling will be compared using performanceevaluation test mentioneq.above:MethodologyNeural Network ForecastingIn this study, HMLP network with MRPE algorithm isused to perform CO concentrations forecasting becausethe network was found to perform better compared toMultilayered Perceptron (MLP), Radial Basis Function(RBF) and Hybrid Radial Basis Function in [9]. Thetheoretical discussion of HMLP network and MRPEalgorithm can be found in [9J.Proceedings ofthe International Conference onRobotics, Vision, Information and Signal Processing ROVISP2005GRAPH CO LEVEL VERSUS NUMBER OFDA'D\.400350300250200150300 -----~----------------------------\~ 250b 200 +-------------------------;r-t---~ 150 +----------:l""""'=-~-r~--____tC'!\,____-__r'----Tt'T-'--­~ 100 +------;;;d---\-:n---r---+----;;>,L---'r--j--------\--j8 50 t-~:;;;::;~7-----....:~o-L---i7,L-----\:II-------~o~:...--.:!..-~-------..:......_-----------'o 50 100NUMBER OF DATAFigure 1 - CO Concentrations versus Number ofDataIn this study, HMLP network will perform COconcentrations forecasting using on-line and off-linetechniques. Usually neural network forecasting will becarried out using off-line model. In this model, the dataset will be divided into two, which consists of trainingset and testing set. HMLP network parameters will onlybe updated using the chosen algorithm in the trainingphase. HMLP network wi11 be trained repeatedly for thetraining data samples until minimum prediction error isachieved. Then, the final network parameters will betested using the independent testing set.network and the remaining 300 data are used to test thefitted model and to calculate index of coefficient (R2).The best three inputs lags obtained from off-linemodelling are shown in Table 1.Table 1- Selected Input Lags for Simulated EnvironmentDataNumber Input~s1 (t-l)(t-2)2 (~I)(t-2)(t-3)(~4)3 (t-l)(t-2)(t-3)(t-4)(t-5)In on-line modelling and forecasting, the HMLPnetwork parameters will be updated for each sample ofCO measurement. In this case, the network parameterswill be always updated to achieve minimum predictionerror. For this model, data are not divided such as off-line technique.In this paper, multiple steps ahead prediction test (MSA)are .carried out to obtain the future value of COconcentrations level. The number of steps ahead to beforecasted has been limited to 8. Forecastingperformance of high prediction level relies on lowerprediction level. If the prediction error for one stepahead is minimum, it will provide a good initialcondition for the higher level prediction [10].Results and DiscussionData from simulated environmentThe simulated environment data set was collected from asimulated environment in a lab. It contains 400 datawere collected with sampling time of 10 seconds. Thedata obtained from simulated environment is shown inFigure 1. The analysis to determine the best input lagsare carried out for off-line modelling. It is veryimportant to determine the best input lags in order toproduce good results in term of forecasting performance.The analysis is limited to 30 input lags, because furtherincreases of input lags are not found to provide anysignificant improvement. Number of hidden nodes usedare 3, for both on-line and off-line model. For off-linemodelling, the first 100 data are used to train theOff-line model is able to produce good results by usingonly 1 epoch, the performance degrades by usingmUltiple epochs. R2 calculated from each set of inputlags using 1 epoch is shown in Table 2. It can be seenthat R2 values varies for different set of input lags. It canbe noted that model seems to perform well by using Lag2. It means that in order to obtain good performance, 4past values of CO concentrations need to be consideredto perform CO concentrations forecasting.able 2- Values Achievedfor 0 "'-line ModeNumber Lag 1 Lag 2 Lag 3of Steps1 0.9728 0.9792 0.97642 0.9263 0.9527 0.94383 0.8598 0.9172 0.89524 0.7734 0.8692 0.82525 0.6683 0.8091 0.72976 0.5472 0.7391 0.61137 0.4122 0.6627 0.47588 0.2639 0.5800 0.3216For on-line modeling, the last 300 data are used tocalculate R2• The analysis to determine the best inputlags for on-line modeling are also carried out. For thecomparison to be fair, the best three inputs lags obtainedfrom off-line model are used to calculate R2 for thismodel.One input lag is chosen from the analysis performed foron-line model, since it is found to give the bestperformance compared to the other lags. The four inputlags used to calculate R2 value are shown in Table 3.753Proceedings of the Intefnational Conference onRobotics, Vision, Information and Signal Processing ROVISP2005Figure 2 - MSEfor Simulated Environment DataTable 3 - Selected Input Lagsfor Simulated EnvironmentDataThe MSE plot for the whole set of data is shown inFigure 2, which indicates that HMLP networkparameters converge rapidly. The MSE plot convergearound 220 data samples.Table 5 • Selected Input Lagsfor Industrial DataNumber Input Lags1 (t-l)(t-2)2 (t-l )(t-2)(t-3)(t-4)(t-5)3 (t-l )(t-2){t-3)(t-4)(t-5)(t-6)In this, section, both models are trained using 100 datasamples only, in order to show performance differencesbetween both models when the networks. are undertrained. From the results obtained, it is proven that on-line model is able to perform well even when thenetwork is trained using small amount of data. Off-linemodel does not give good prediction for multiple step~ahead forecasting. It can be concluded that the numberof training data used has a very strong influence towardsthe off-li~emodel performance. 'Industrial Dataforecasting performance in terms of R2 test. Forcomparison using the best input lags obtained from-off~line model, it can be concluded that on-line model isfound to perform slightly better compared to off-linemodel. Besides that, it performs even better by using thebest input lags obtained from the analysis using on-linemodel. The differences between both models becomemore noticeable when the number of steps ahead 'forforecasting is increased. The maximum differencesachieveq by R2 value using the same lags are around 0.2and 0.3.The industrial data set waS obtained from MalaysianEnvironmental Department (ASMA): These datacontains the average hourly of CO measurements. Thesedata were obtained from industrial area in Penang. 1000data samples are used to perform the task for this paper.The Industrial Data is shown in Figure 3. 1000 data are·used to perform CO forecasting, it contains .averagehourly CO concentrations measurement. For off-linemodeling, .the fIrSt 600 data are used to train the networkwhile the remaining 400 data are used to test the datasamples. The testing data samples are used. to calculateR2• The analysis to determine the best input lags areperformed in· order to obtain the best results from themodel. The analysis is limited to 30 input lags becausemore input lags do not give any significant improvementin terms of R2 test. The best three inputs lags obtainedfrom off-line modelling are shown in Table 5.~ values calculated for each set of inputs are shown inTable 6. For industrial data, the off-line models aretrained using 3 epochs, because the model does notproduce good results by 'using 1 epoch. The R2 valuesachieved with 1 epoch using the selected input'lags areshown in Table 6, while the results obtained using 3epochs are shown in Table 7., :, ;; ,: :2Number InputLa~s1 (t-l)(t-2)2 (t-l)(t-2)(t-3)(t-4) .3 (t-l )(t-2)(t-3)(t-4)(t-5)4 (t-l)(t-2)(t-3)(t-4)(t-5)....(t-ll)Table 4 - R Values Achieved for On-line ModelNumber Lag 1 Lag 2 Lag 3 Lag~of Steps1 0.9764 0.9800 0.9791 .0.98102 0.9387 ·0.9550 0.9509 0.95523 0.8854 0.9i90 0.9146 0.91954 0.8171 0.8700 0.8661. 0.870i.5 0.7352 0.8100 0.8071 0.81006 0.6430 0.7360 0.7400 0.74477 0.5437 0.6658 0.6671 0.67568 0.4384 0.5931 0.5892. 0.5997The R2 values achieved for the input lags used is shown. in Table 4; First; the results are compared using the sameinput lags as the off-line model. In that case, Lag 2 gavethe best results compared to the others. From theanalysis shown in Table 4, it can be concluded that Lag4 is found to perform even better compared to Lag 2.This means that only 11 past CO concentrations valueneed to be obtained in order to produce good accuracy interms of forecasting performance.From the results shown using simulatedenvironment data, it can be concluded that both on-lineand off-line model using HMLP network produce good754Proceedings ofthe International Conference onRobotics, Vision, Information and Signal Processing ROVISP2005GRAPH CO LEVEL VERSUS NUMBER OF DATA30~ 2520.....:I 15~.....:I)(}8 500 )(){) 200 3(){) 400 500 600 700 SOO 9(){) }(}ONUMBER OF DATAdLa21'. bl 8 la e-nyut gs use for Industrial DataNumber InputLa~s1 (t-1)(t-2)2 (t-1)(t-2)(t-3)(t-4)(t-5)3 (t-1)(t-2)(t-3)(t-4)(t-5)(t-6)4 (t-1)(t-2)(t-3)(t-4)(t-5) ....(t-21)Table 9. R Values Achieved or On-line ModelNumber Lag 1 Lag 2 Lag 3 Lag 4of Steps1 0.7354 0.7300 0.7165 0.73372 0.4638 0.4558 0.4239 0.51523 0.2685 0.2637 0.1122 0.38554 0.1367 0.1379 0.0544 0.30715 0.0286 0.2637 0.1122 0.23876 -0.054 0.0128 -0.176 0.19517 -0.123 -0.063 -0.250 0.1838 -0.171 -0.025 -0.313 0.1809The MSE plot for 1000 data samples is shown in Figure4. The MSEconverges to an acceptable value after about500 data samples. It indicates that HMLP networkrequires 500 data samples for off-line modelling.It can be concluded that the R2 values achieved by on-line and off-line models are only good for one stepahead forecasting. The R2 values obtained are found todrop drastically for two steps ahead forecasting with thedifferences between 0.2 and 0.3. Overall, both, modelsdo not give good prediction after 3 steps. This could beFirst, the results are compared using the same input lagswith off-line model. In that case, Lag 2 gave the bestresults compared to the others. From the analysis shownin Table 9, it can be concluded that Lag 4 is found toperform even better than Lag 2. This means 21 past COconcentrations value need to be considered in order toproduce better accuracy in terms of forecastingperformance compared to others.IPOCNumber of Lag 1 Lag 2 Lag 3Steps1 0.7072 0.6840 0.65872 0.3972 0.3944 0.29683 0.1541 0.2067 -0.02224 -0.0218 0.0620 -0.27815 -0.1683 -0.0297 -0.50056 -0.2581 -0.0739 -0.65267 -0.3271 -0.0971 -0.83178 -0.3570 -0.1044 -0.9703~poc sNumber of Lag 1 Lag 2 Lag 3Steps1 0.7251 0.7168 0.71742 0.4464 0.4392 0.41823 0.2423 0.2707 0.17624 0.0958 0.1717 -0.00775 0.0189 0.1237 -0.1726 -0.0940 0.1052 -0.27777 -0.1427 0.0954 -0.37418 -0.1745 0.0872 -0.4618Table 7- KI Values AchievedforOff-line Model Using 3E hFigure 5 - CO Concentrations versus Number ofDataTable 6. If Values Achievedfor Off-line Model Using I to the input lags obtained from off-line model. The fourE h input lags used to calculate R2 value are shown in Table8. The R2 value achieved for the input lags used isshown in Table 9.For on-line modelling, the last 400 data are used tocalculate R2 value. Analysis to determine the best inputlags for on-line modeling are carried out. For thecomparison to be fair, the best three inputs lags obtainedfrom off-line model are used to calculate R2 value. Onebest input lag is chosen from analysis performed for on-line model, since it is found to perform better compared. From the results shown in Table 6 and Table 7, it can benoted that off-line model using 3 epochs is found toperform better compared to single epoch. Off-line modelusing Lag 2 gave better results compared to the others.This means that 5 past CO concentrations value need tobe considered in order to produce better accuracy offorecasting performance.755Proceedings ofthe lntefnational Conference onRobotics, Vision, Information and Signal Processing ROVISP2005due to under sampling of CO concentration-measuremeJ;lts,--since-the data obtained from ASMA- - -were sampled hourly. Besides that, Figure 5 shows thatCO concentrations level are very dynamic because theplot shows that CO level fluctuates heavily. The on-linemodel gives better results compared to off-line modelfor one and two step ahead forecasting. The differencesbetween both the models are more noticeable when thenumber of forecasting step' increased. On-line modelperforms better even though the off-line model used forcomparison applied multiple epochs. For industrial data,on-line model is found to perform better compared tooff-line model. .. ,~,_. ~_.:. ~~ _ • ..1_. Kl- _. ~~"..;••• ;._. ~ •• .i, _ ,.~ _ ~~._ .;:_~~t._~ '" ..... , •. '_ ._. ,_! !! ! f 1, ,; :;""j--" '!"; -,' .;".... '$""1' '" 'r"02664110164216 270324378~ 486 538592646700 754~860914 938DATAFigure 4 - MSEPlotfor Industrial DataConclusionsThis study proves that HMLP network using ModifiedRecursive Prediction Error Algorithm (MRPE) can beused to perform CO concentrations forecasting. Both on-line and off-line models provide good results in terms offorecasting perfo~hce. However, on-line model is. found to perform better compared to off-line modelusing both sets of data due to its learQing capability.Besides that, sampling time used has significantcontribution towards forecasting performance. If thesampling time is very large, the network will not be able... to produce good results for multi steps aheadforecasting. Overall, forecasting performance dependson the dynamic of the data setReferences[If Carbon- Monoxide Alert, 2001. Citing InternetSources URL http://www.homesafe.comlcoalen.2001.[2] Sharma, P., Khare, M., 2000. Real- time Predictionof Extreme Ambient carbon MonoxideConcentrations due to Vehicular Exhaust Emissionsusing Univariate Linear Stochastic Models.Transportation Research: Part D, Vol.5, No.l:59-69.[3] Sharma, P., Khare, M., 2001. Short-term, Real- time-Prediction of-Extreme Ambient carbon MonoxideConcentrations due to Vehicular Exhaust Emissionsusing -Transfer Function Noise Model,Transportation Research: Part D, VoI.6, No.2:l41-146.. [4] Zickus, M. & Kvietkus, K., 1998. Urban AirPollution Forecast based on the Gaussian imdRegression Models. In Proceedings of SixthInt~nationalConference on Air Pollution, 515-523." [5] Comrie, AC. & Diem, J.E., 1999. Climatology andForecast Modelling in Phoenix, Arizona.At.nospheric Environment, Vol.33, No.30:5023-5036.[6] Moseholm, .L" Silva, J. & Larson, T., 1996.Forecasting Carbon Monoxide Carbon MonoxideConcentrations Near a Sheltered Intersection usingVideo Traffic surveillance and Neural Networks..Transportation Research D, Vol. 1, No.1: 15-28.[7] Patricio, P., Trier, A. & Reyes, J., 2000. Predictionof PM2.5 Concentrations Several Hours in Advanceusing. Neural networks in Santiago, Chile:Atmospheric "Environment, Vo1.34, No.8:il89-1196.[8] Lu, W.Z., Wang, W.J., Wang, X.K., Yan, S.H. &Lam, i.C., 2004. Potential Assessment of a NeuralNetwork Model with PCAlRBF approach forForecasting Pollutant Trends in Mong Kok urbanair, Hong Kong. Environmental Research, Vo1.96,No.l:79-87.[9] Mashor, M.Y., 2000. Hybrid Multilayered·Perceptron Networks, International Journal ofSystem Science,Vol.31:771-785. .[lO]Berger, C.S., 1995. Recursive Single Layer Nets forOutput Error Dynamic Models, IEEE TransactionsOn Neural Networks, Vol.5: 508-511.756

Climatology and Forecast Modelling in Phoenix, Arizona. At.nospheric Environment,

Forecasting Carbon Monoxide Carbon Monoxide Concentrations Near a Sheltered Intersection using Video Traffic surveillance and Neural Networks..

Hybrid Multilayered· Perceptron Networks,

Monoxide Alert,

Potential Assessment of a Neural Network Model with PCAlRBF approach for Forecasting Pollutant Trends in Mong Kok urban air, Hong Kong. Environmental Research,

Real- time Prediction of Extreme Ambient carbon Monoxide Concentrations due to Vehicular Exhaust Emissions using Univariate Linear Stochastic Models. Transportation Research: PartD,

Short-term, Real- time -Prediction of-Extreme Ambient carbon Monoxide Concentrations due to Vehicular Exhaust Emissions using -Transfer Function Noise Model, Transportation Research: Part D,

Urban Air Pollution Forecast based on the Gaussian imd Regression Models.

On-Line  Modelling  And Forecasting Of Carbon Monoxide Concentrations Using Hybird  Multilayered  Perceptron  Network.

http://eprints.usm.my/8617/1/On-Line_Modelling_and_Forecasting_of_Carbon_Monoxide_Concentrations_Using_Hybrid_Multilayered_Perceptron_Network_(PPKejuruteraan).pdf

On-Line Modelling And Forecasting Of Carbon Monoxide Concentrations Using Hybird Multilayered Perceptron Network.

Abstract

Similar works

Full text

Available Versions

Repository@USM