This paper presents a non-conventional dynamic neural network that was designed for real time prediction of NOx at the coal powder power plant Mělnik 1, and results on real data are shown and discussed. The paper also presents the signal preprocessing techniques, the input-reconfigurable architecture, and the learning algorithm of the proposed neural network, which was designed to handle the non-stationarity of the burning process as well as individual failures of the measured variables. The advantages of our designed neural network over conventional neural networks are discussed

Bukovský, Ivo

Kolovratník, Michal

Acta Polytechnica

English

CTU Open Journal Systems (Czech Technical University, Prague / České vysoké učení technické v Praze)

Acta Polytechnica Vol. 52 No. 3/2012A Neural Network Model for Predicting NOx at the Meˇln´ık 1Coal-powder Power PlantIvo Bukovsky´1, Michal Kolovratn´ık21Czech Technical University in Prague, Faculty of Mechanical Engineering, Department of Instrumentation and ControlEngineering, Technicka´ 4, 166 07 Prague, Czech republic2Czech Technical University in Prague, Faculty of Mechanical Engineering, Department of Energy Engineering,Technicka´ 4, 166 07 Prague, Czech republicCorrespondence to: ivo.bukovsky@fs.cvut.czAbstractThis paper presents a non-conventional dynamic neural network that was designed for real time prediction of NOx atthe coal powder power plant Meˇln´ık 1, and results on real data are shown and discussed. The paper also presents thesignal preprocessing techniques, the input-reconﬁgurable architecture, and the learning algorithm of the proposed neuralnetwork, which was designed to handle the non-stationarity of the burning process as well as individual failures of themeasured variables. The advantages of our designed neural network over conventional neural networks are discussed.Keywords: dynamic neural networks, prediction, NOx emissions, signal processing.1 IntroductionNeural networks (NN) are a popular and widely studied real data-driven nonlinear modeling tool for compli-cated systems where mathematical-physical analysis is unavailable for deriving a model. Unlike analytical orlinear models, NNs are black-box models, or sometimes gray-box models, that require proper design of theirmathematical architecture and an eﬃcient learning algorithm. For the principles of fundamental neural net-works we may refer, e.g., to [1], and we may refer to less recent reviews [2, 3] for studies of NNs in energeticprocesses. For more recent works, including studies of conventional NN in energetic processes, we may refer topapers [5–8], which deal with computational intelligence tools (neural networks, genetic algorithms) focused onbiomass combustion. The study of non-conventional neural architectures for modeling steady state hot steamturbine data and for modeling a large scale energetic boiler can be found in [9], where the advantages of astatic quadratic neural unit (QNU, [1, 4]) and a special quadratic neural network [9] over conventional multi-layer perceptron neural networks (MLP) are demonstrated, with reference to the overﬁtting and local minimaproblem, which are typical drawbacks of MLP (even with a single hidden layer NN). The advantage of QNUis its nonlinear input-output mapping, while this neural model is linear in its parameters [10] (unlike MLP).This allows us to monitor and maintain adaptation stability by a comprehensible evaluation of the eigenvaluesof the weight update system [10] that oﬀer promising opportunities for adaptive monitoring, modeling, andprocess optimization by adaptive nonlinear control. QNU can be seen as a component of higher-order neuralnetwork (HONN), sometimes also referred to as a polynomial neural network (PNN). The origins of these neuralnetworks can be traced back to works [11–14], while the concept of the standalone higher-order neural units(HONUs) as a building component of HONN can be found in [1] and in [4]. The fundamental gradient-basedlearning rules for training dynamic neural networks are known as Real-time Recurrent Learning (RTRL) [15]and Back-Propagation Through Time (BPTT) [16, 17]. These algorithms can be made comprehensible andpractically useful for real-time computations.In this paper, we present the resulting neural network architecture that has been designed and tested for NOxprediction for a pulverized coal ﬁring boiler at the power plant “Elektra´rna Meˇln´ık 1 (EME 1)”; the nominalsteam load of this boiler is 250 tons per hour. The goal is to design and test a model that does not involvemeasured O2 or CO in its input and that can potentially be used for optimizing the energetic process regardingNOx and CO emissions of the pulverized ﬁring boiler at EME 1. The resulting discrete-time dynamic (recurrent)neural network merges the concept of a conventional recurrent (MLP) neural network with QNU [4, 9]. The17Acta Polytechnica Vol. 52 No. 3/2012Figure 1: The data preprocessing before each reconﬁguration and retraining of the neural networkdata pre-processing and retraining strategy is described in the next section, and that in turn is followed by themathematical notation of the neural architecture that led to the results shown in the section on discussion.2 Data preprocessing and network trainingThe NOx dynamics of the pulverized boiler is highly nonstationary, due to varying technical conditions of theboiler, varying quality of the coal powder, and also because of the measurement outages that occur quite oftenon an hourly basis. It was therefore not possible to obtain a neural network model that would reliably predictthe NOx emissions from the long term data. To handle the non-stationary nature of the boiler in EME 1, wearrived at the data preprocessing technique that is sketched in Figure 1, whereU(k) is a matrix of recent history(a retraining window) of all measured input variables (excluding measured O2, NOx, and CO) at a referencetime k that is particularly given as followsU(k) =⎡⎢⎢⎢⎢⎢⎣u1(k −Ntrain+ 1) . . . u1(k − 1) u1(k)u2(k −Ntrain+ 1) . . . u2(k − 1) u2(k).... . .......un(k −Ntrain+ 1) . . . un(k − 1) un(k)⎤⎥⎥⎥⎥⎥⎦ . (1)The measured input variables inU(k) are the primary, secondary, and tertiary air valves, and also optionally thesteam load or the air ﬂow before the ventilator (in total n = 18, 19, 20 variables) excluding O2, NOx, and CO.The principal component analysis (PCA) block is an application of PCA to the linearly correlated variables,so the number of input variables in Upca(k) is m < n, which importantly decreases the computation loadwhile it maintains information in the measured input data (note, Figure 1 shows only a simpliﬁed sketch, whiledetailed implementation of PCA that beneﬁts from process knowledge of the pulverized boiler at EME 1 may beprovided on the basis of an oﬃcial request for [19]). The structure of the resulting input data matrix Upca(k)that is used as the neural network input (after the preprocessing shown in Figure 1) is as followsUpca(k) =⎡⎢⎢⎣upca1(k −Ntrain+ 1) . . . upca1(k − 1) upca1(k).... . .......upcam(k −Ntrain+ 1) . . . upcam(k − 1) upcam(k)⎤⎥⎥⎦ . (2)The presented data pre-processing technique removes variables with measurement outages. Principal componentanalysis results in a lower computational load, because of the reduced number of external inputs into the neuralnetwork (m < n). PCA has a ﬁltering eﬀect, and also contributes to more accurate calculations of matrixinversion with the BPTT training technique by reducing redundant and linearly correlated data.3 Neural network for NOx predictionThis section describes the mathematical notation of the designed neural network for NOx prediction. Thisneural network is a discrete-time recurrent architecture, i.e., a non-linear diﬀerence equation system, composedof a recurrent hidden layer of conventional sigmoid neurons and with an output quadratic neural unit withfeedbacks also from the output to its input. In particular, the neural network predictive model is given as18Acta Polytechnica Vol. 52 No. 3/2012follows. The window of external inputs for retraining the network at reference time k are the pre-processedmeasured variables Upca(k), as given in (2) and in Figure 1. The external inputs that enter the neural networkfor ns samples ahead prediction at time k are in the last column of Upca(k), as followsupca(k) =[upca1(k) upca2(k) . . . upcar(k)]T, (3)where k is a reference sample time index and r is the dimension of the reduced vector of all measured externalinputs by the PCA method. The input vector to the hidden layer of the neural network is given in (4) asx(k) =[yn(k + nya) . . . yn(k − nyb) upca(k + nua)T . . . upca(k − nub)T ξ(k)]T, (4)where yn(.) are step-delayed neural outputs; nya, nyb, nua, and nub are input conﬁguration parameters; andξ(k) is the step delayed feedback of the hidden layer outputs. The output of the hidden sigmoidal layer ξ(k+1)(6) is calculated using the hidden layer weight matrix W (6) and using the classical sigmoid function (5), asfollowsφ(ν) =21 + e−ν− 1, (5)where ξ(k + 1) is augmented with a unit asξ(k + 1) =[1φ(W · x(k))], (6)where the unit allows the hidden layer (ﬁrst column of W) and also the QNU (v0,0 in (7)) for biases, so theneural output is calculated by a quadratic neural unit [1, 4, 9, 10], using (3)–(6), as followsyn(k + ns) =∑i=0∑j=ivi,j · ξi(k + 1) · ξj(k + 1). (7)The proposed dynamic neural network has purposely designed properties that are worth mentioning and ex-plaining. The hidden recurrent layer of neurons, which is calculated in (6) as φ(W · x(k)), reduces cognitively(by training) the number of already PCA preprocessed neural inputs, and thus (6) results in the augmentedvector of state variables ξ(k+1) that are fed both forward to the QNU and also back to the network input x(k),as in (4). Without the ﬁrst hidden layer, the number of input variables inputted directly into QNU would stillbe too large for the given 1-minute sampling period, as we feed an approximately twelve-minute history of eachPCA preprocessed variable into the network input, i.e. nya − nyb = 12 and also nua − nub = 12 (the estimatedtime constant of this pulverized ﬁring boiler has been speciﬁed by experts as approximately 12 minutes). Also,the ﬁrst layer (6) plays a ﬁltering role due to its step delayed feedback to the network input (4), and its recurrentfeedback naturally calls for training by the Backpropagation Through Time method (BPTT) [15–17], which isa powerful and eﬃcient and yet practical optimization method, as it can be achieved by a combination of agradient descent rule and the Levenberg-Marquardt algorithm [18]. The sigmoid function ϕ(.), which is usuallyconsidered as a main nonlinearity of conventional neural networks, has another importance for this dynamicnetwork, because the major nonlinearity is provided by the QNU [9, 10, 18]. The sigmoid function ϕ(.) limitsthe output of the hidden layer into the given range of values (−1,+1) that importantly assures stability of thehidden layer (as of a discrete time dynamic system, and this could not be so simply assured for continuous-timeNNs). Then, there are always limited values entering the QNU, so its output is also naturally limited; thus,the stability of the state variables and of the output of the proposed neural network is naturally assured bypreserving the sigmoid function in hidden neurons. As regards the stability of the learning algorithm, and thusits convergence, we proposed a novel approach to weight-update stability for gradient descent training of QNUin [10], and this approach is applicable to both static and dynamic QNU, and also to the hidden-layer weightsystem of this network for NOx prediction.4 Results and discussionThis section shows the results of 3-minute predictions of NOx emissions (in fact the 3-minute ﬂoating averages)of the pulverized ﬁring boiler at EME 1 by the proposed neural network (ns = 3, sampling 1 minute). Therecurrent network does not include measured O2, NOx, or CO on its input; the introduction of NOx as a measuredexternal input resulted in a prediction failure; the model learned to follow blindly the previous measured NOx,19Acta Polytechnica Vol. 52 No. 3/2012Figure 2: NOx prediction by the neural network (1)–(7) with re-conﬁgurations and re-training (Figure 1)Figure 3: Detail from Figure 2 — a good predictionFigure 4: Detail from Figure 2 — a bad prediction due to outliers in the training datawhich is a typical problem with the improper use of neural networks for complicated systems. The permanentcomputation run of prediction for 24 days, with one-minute sampling and retraining every 30 minutes, is shownin Figure 2.Figure 2 shows superimposed 24-day recordings of measured NOx (thick line) and the three-minute prediction(ns = 3) of NOx (bold line) (the three-minute ﬂoating average of NOx is predicted); the neural network isretrained every 30 minutes with 5 hours of the very last measured data (one-minute sampling, the model inputexcludes measured O2 , NOx, CO). The network (1)–(7) was retrained every 30 minutes by the back propagationthrough time algorithm [18] with blindly selected most recent history of 298 samples (5 hours) of the measuredprocess variables. Each retraining took less than 3 minutes of real computation time in Matlab on a PC (Win7,i7), and this is practical for real time retraining implementation.The good performance of NOx prediction is apparent from the details in Figure 3. Also, Figure 3 shows atemporary measurement outage (∼ 15 minutes) after sample k = 2.82E + 4; the output of the dynamic neural20Acta Polytechnica Vol. 52 No. 3/2012network substitutes the measurement outage; the good neural network prediction depends on availability ofgood retraining data in this observed period. However, this kind of NOx outage aﬀects retraining, see Figure 4.The prediction accuracy and prediction reliability for NOx prediction depends signiﬁcantly on the retrainingdata (here, the last 298 samples before each predicted value). The impact of NOx outliers is apparent if wecompare the prediction details in Figure 3 and Figure 4, and it is clear that another signal processing techniquefor selecting the retraining data needs to be involved in order to avoid NOx outliers in the retraining data; theneural network fails in prediction after k = 3.122E+ 4 because of the poor retraining data and also because ofthe outliers at k = 3.12E + 4. The prediction becomes correct again for k > 3.133E + 10, because the relatedretraining data already does not include the outliers.5 ConclusionsWe designed and tested a non-conventional recurrent neural network for predicting the NOx emissions of apulverized ﬁring boiler without using O2, NOx, or CO on the model input. The proposed method handlesprocess non-stationarity by frequent retraining, and it handles the outages of input process variables by inputdata preprocessing (but not yet the outages of the predicted NOx itself); it is assumed that this can be resolvedby automatically supervised selection of the training data where NOx outliers do not appear, and by avoidingunnecessary retraining.AcknowledgementThis work has been supported by grant MPO FR-TI1/538, and in part by grant SGS10/252/OHK2/3T/12.References[1] Gupta, M. M., Liang, J., Homma, N.: Static and Dynamic Neural Networks: From Fundamentals toAdvanced Theory. IEEE Press and Wiley-Interscience, John Wiley & Sons, Inc., 2003.[2] Kalogirou, S. A.: Artiﬁcial intelligence for the modeling and control of combustion processes: a review,Progress in Energy and Combustion Science, 29, 2003, p. 515–566, Elsevier. ISSN 0360-1285.[3] Mellit, A., Kalogirou, S. A.: Artiﬁcial intelligence techniques for photovoltaic applications: A review,Progress in Energy and Combustion Science, 34, 2008, p. 574–632, Elsevier. ISSN 0360-1285.[4] Bukovsky, I., Bila, J., Gupta, M. M., Hou, Z.-G., Homma, N.: Foundation and Classiﬁcation of Nonconven-tional Neural Units and Paradigm of Nonsynaptic Neural Interaction, in Discoveries and Breakthroughs inCognitive Informatics and Natural Intelligence. in the ACINI book series ed. by Yingxu Wang, Universityof Calgary, Canada : IGI Publishing, Hershey PA, USA, 2009. ISBN 978-1-60566-902-1.[5] Pitel’, J., Mizˇa´k, J.: Approximation of CO/Lambda Biomass Combustion Dependence by Artiﬁcial Intelli-gence Techniques. In Annals of DAAAM for 2011 & Proceedings of the 22nd International DAAAM Sym-posium, Vienna, Austria, 23–26th November 2011. Vienna : DAAAM International, 2011, p. 0143–0144.ISBN 978-3-901509-83-4, ISSN 1726-9679.[6] Mizˇa´k, J., Pitel’, J.: Using Artiﬁcial Neural Networks for Biomass Combustion Process Control. InProceedings of 2nd International Seminar “System Analysis, Control and Information Processing”, Di-vnomorskoje, Russia. [CD-ROM]. Rostov on Don: Don State Technical University, 2011, p. 343–348. ISBN978-5-7890-0666-5.[7] Pitel’, J., Borzˇ´ıkova´, J., Mizˇa´k, J.: Biomass Combustion Process Control Using Artiﬁcial IntelligenceTechniques. In Proceedings of XXXVth Seminar ASR’2010 “Instruments and Control”. Ostrava : VSˇB-TUOstrava, 2010, p. 317–321. ISBN 978-80-248-2191-7.[8] Hosˇovsky´, A.: Genetic Optimization of Neural Networks Structure for Modeling of Biomass-Fired BoilerEmissions. Journal of Applied Science in Thermodynamics and Fluid Mechanics, Vol. 9, No. 2, 2011, p. 1–6.ISSN 1802-9388.[9] Bukovsky´, I., Lepold, M., Bı´la´, J.: Quadratic Neural Unit and its Network in Validation of Process Dataof Steam Turbine Loop and Energetic Boiler, WCCI 2010, IEEE Int. Joint. Conf. on Neural NetworksIJCNN, Barcelona, Spain, 2010.21Acta Polytechnica Vol. 52 No. 3/2012[10] Bukovsky´, I., Bı´la´, J., Noriasu, H., Rodriguez, R.: Prospects of Gradient Methods for Nonlinear Control,Automatiza´cia a riadenie v teo´rii a praxi ARTEP 2012, Slovakia 2012. ISBN 978-80-553-0835-7.[11] Ivakhnenko, A. G.: Polynomial Theory of Complex Systems, IEEE Tran. on Systems. Man and Cybernetics.Vol. SMC-1, 4, 1971, p. 364–378.[12] Nikolaev, N. Y., Iba, H.: Learning Polynomial Feedforward Neural Network by Genetic Programming andBackpropagation, IEEE Trans. on Neural Networks, Vol. 14, No. 2, March, 2003, p. 337–350.[13] Taylor, J. G., Commbes, S.: Learning higher order correlations, Neural Networks, 6, 1993, p. 423–428.[14] Kosmatopoulos, E., Polycarpou, M., Christodoulou, M., Ioannou, P.: High-Order Neural Network Struc-tures for Identiﬁcation of Dynamical Systems, IEEE Trans. on Neural Networks, Vol. 6, No. 2, March 1995,p. 422–431.[15] Williams, R. J., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks,Neural Comput., Vol. 1, 1989, p. 270–280.[16] Werbos, P. J.: Backpropagation through time: What it is and how to do it, Proc. IEEE, Vol. 78, No. 10,Oct. 1990, p. 1 550–1 560. ISSN 0018-9219.[17] Pearlmutter, B. A.: Gradient calculation for dynamic recurrent neural networks: a survey. IEEE Transac-tions on Neural Networks, 6, 5, 1995, 1 212–1228. doi: 10.1109/72.410363.[18] Gupta, M. M., Bukovsky´, I., Noriasu, H., Solo, M. G., Hou, Z.-G.: Fundamentals of Higher Order NeuralNetworks for Modeling and Simulation, In Artiﬁcial Higher Order Neural Networks for Modeling andSimulation, ed. M. Zhang, IGI Global, 2012, (accepted, to appear in 2012).[19] Bukovsky´, I., Krˇehl´ık, K.: Testy neuronove´ho modelu kotle elektra´rny Meˇln´ık I. Vy´zkumna´ zpra´va cˇ. 6 –ZI00069/E06. U´stav prˇ´ıstrojove´ a rˇ´ıd´ıc´ı techniky, Fakulta Strojn´ı, CˇVUT v Praze, 2011.22

A Neural Network Model for Predicting NOx at the Mělník 1

Abstract

Similar works

Full text

Available Versions

CTU Open Journal Systems (Czech Technical University, Prague / České vysoké učení technické v Praze)

Directory of Open Access Journals