A Matlab Toolbox is developed for the design, construction and validation of grey-box neural network models. This toolbox, available in www.diinf.usach.cl=gacuna has been tested in simulations with a continuously stirred reactor process. The grey-box model performs well for validation data with 5% additive gaussian noise for one-step-ahead (OSA) and model-predictive-output (MPO) estimations

Acuña, Gonzalo

Pinto, Erika

Agora University Editing House: Journals

International Journal of Computers, Communications & ControlVol. I (2006), No. 2, pp. 7-14Development of a Matlabr Toolbox for the Design of Grey-Box NeuralModelsGonzalo Acuña, Erika PintoAbstract: A Matlab Toolbox is developed for the design, construction andvalidation of grey-box neural network models. This toolbox, available inwww.diinf.usach.cl/gacuna has been tested in simulations with a continuously stirredreactor process. The grey-box model performs well for validation data with 5% ad-ditive gaussian noise for one-step-ahead (OSA) and model-predictive-output (MPO)estimations.Keywords: Grey-Box Model, Neural Networks, One-Step-Ahead estimation, Multi-ple Prediction Output, Time variant parameter identification.1 IntroductionIn the development of dynamic system models it is best to take advantage of the a priori knowledgeof a process, generally expressed in terms of a set of ordinary differential equations which representmass or energy balances. In complex biotechnological processes, the most difficult task is the modelingof the time-varying parameters, such as specific kinetics. In order to address this problem [1], proposedthe use of grey-box models which combine a priori knowledge expressed in terms of a phenomenolog-ical, or white-box model, with a black-box model such as neural network. These models have provedto be satisfactory for dynamic systems, they have better generalization characteristics, and they can beidentified with a smaller amount of data [1]. [2] classified these grey-box models into two principal cate-gories: those which deliver intermediate values (of parameters or variables) for use in phenomenologicalmodels (serial grey-box models), or those in parallel with the dynamic model, adjusted to compensatefor modeling errors (parallel grey-box models). [3] showed that the series strategy resulted in grey-boxmodels with superior results. More recently [4] and [5] have employed and analyzed this type of modeldemonstrating their performance and their use in complex processes.Matlabr is well known as a completely integrated application development environment, orientedto projects that involve complex mathematical calculations and graphic visualizations. This softwarehas a large variety of toolboxes which are specialized packets which can carry out different functionsaccording to the area of development, for example; optimization, image processing, neural networks,simulations and statistics among others. Thus, the present work deals with the creation of a group offunctions integrated in a Matlabr Toolbox which allows the development of an Grey-Box Neural Model(GBNM) for complex systems in general.The present document is organized as follows: Section 2 details some of the aspects of the GBNMsused, Section 3 examines the design and construction of the Toolbox; Section 4 looks at the applicationsof the Toolbox; and Section 5 outlines the conclusions reached.2 Grey-Box Neuronal ModelsAs mentioned previously GBNMs take advantage of the combination of the a priori knowledge of agiven process -expressed in terms of a set of differential equations that represent the first principles thatgovern the process- with neural networks. The latter are responsible for modeling the interaction betweenvariables that are relevant to the system, and certain time-varying. It is a well established fact that neuralnetworks are capable of approximating non-linear functions. In particular, it has been demonstrated thatperceptrons, with only one hidden layer and an adequate number of neurons in their internal layer, areCopyright c© 2006 by CCC Publications8 Gonzalo Acuña, Erika Pintouniversal approximators [6].For the purposes of the present work it is important to distinguish between two training modes forneural networks included in GBNMs. The first type, also known as the direct learning mode [5], uses theerror generated at the output of the neural network for the correct determination of its weights (Figure1).Figure 1: Grey-Box Neural Model in its direct learning mode.The second type corresponds to an indirect mode by which the error generated at the output of theGBNM is used for the training of the neural network [5] (Figure 2).Figure 2: Grey-Box Neural Model in its indirect learning mode.The Toolbox developed in the present work is based on the direct learning mode of the neural net-work. The neural networks used are multi-layered perceptrons with only one hidden layer. The trainingalgorithm is error retropropagation combined with a Levenberg-Marquardt optimization.The validation of the results obtained is carried out with tests that consist in evaluating the error pro-duced when using the GBNM for a One Step Ahead (OSA) prediction, and for Model Predictive Output(MPO) prediction, [7]. The error indices used are three: the Root Mean Square (RMS) error, the RelativeStandard Deviation (RSD) and the Adecuation Index (IA), which are presented below [7]:RMS =√∑ni=1(oi− pi)2∑ni=1 o2i; RSD =√∑ni=1(oi− pi)2N; IA = 1− ∑ni=1(oi− pi)2∑ni=1(|o′i|+ |p′i|)2Where oi and pi are the observed and predicted values respectively, in time i, and N is the totalnumber of data. p′i = pi−om and o′i = oi−om, where om is the median value of the observations.3 Development of the Matlab ToolboxThis Toolbox consists of two principal parts: a Neural Network and serial Grey Box Model, eitherwith OSA or MPO estimates.Development of a Matlabr Toolbox for the Design of Grey-Box Neural Models 9The methodology used corresponds to the Modified Cascade Model which has the following stages:• Definition of requirements.• System Design.• Implementation and testing of units.• Integration and testing of system.The details of the methodology can be found in [8].3.1 Neural NetworkThis is the first part of the Toolbox, and its principal function is to create and train the network thatthe serial Grey-Box Model uses for estimating the unknown parameter or parameters which are thencombined with the phenomenological part of the model. For this reason it is necessary to run this part ofthe program before running the serial Grey-Box Model.The principal functions of the neural network are the following:• Verify the existence of data: This function’s role is to verify the existence of the data necessary forthe functioning of the neural network.• Verify dimensions: This function verifies that the quantity of input output data is equal.• Verify layers: This function verifies that the total number of neurons in hidden layers is correct.• Divide data: This function divides input and output data that have been entered by the user intotraining and validation data, (70% and 30% respectively).• Create and train the network: This function creates and trains the neural retropropagation networkbased on the data input by the user. This is carry out using the Matlabr neural network Toolboxfunctions. For the creation of the neural network the “trainlm” function is used which applies theLevenberg-Marquardt method for optimizing weights.• Simulate network outputs: This function simulates network outputs for the purpose of validation.• Graph network inputs: This allows the visualization of the behavior of the entered input.3.2 Serial Grey Box ModelThis is the second part of the Toolbox which allows the development of general serial Grey-BoxModels either with OSA or MPO estimations.The principal functions of the Serial Grey-Box Model applied either to OSA or MPO estimations arethe following:• Verify the existence of the files: This function verifies the existence of the files necessary for thefunctioning of the serial Grey-Box Model.• Verify the existence of data: Among these data are the number of iterations, the initial values forthe status vector, the value of delta t, and where applicable to input u and the real values of thestatus vector variables for each instant of time used.10 Gonzalo Acuña, Erika Pinto• Verify dimensions: This function verifies that the dimensions of the input vector be the same asthe number of iterations. For estimating OSA the quantity of real values of the status variablesmust be equal to the number of iterations. For MPO it only returns the results for the output of thesystem, but cannot evaluate said output with real data.• Develop Model: This function is in charge of developing the Grey-Box Model which consists ofestimating the unknown parameter or parameters by using the neural network created and trainedpreviously in order to develop the different equations entered.• Return different graphs: Graphs the inputs of the system, each of the components of the statusvector for the total number of iterations, the validation of the unknown parameter or parameters,and the output of the system together with its validations.• Calculate the error indices: Calculates RSD, RMS and IA for the output.• Display results: returns the last iteration both for the status vector and the output vector, and forthe error indices that correspond to the unknown parameter or parameters or the output or outputsof the system.4 Aplications and Results4.1 Description of the processIn the simulation of a CSTR process a first order exothermic reaction is used [9], in which the inputsto the system correspond to the temperature of the cooling sleeve and the system output corresponds tothe degree of completion of the reaction. The systems is described by the following status equations:x′1 =−x1−Da · (1− x1) · e(x21+ x2γ)(4.1)x′2 =−x2−B.Da · (1− x1) · e(x21+ x2γ)+β .(u− x2) (4.2)y = x1 (4.3)Where: x1 : Degree of completion of the reaction; x2 : Adimensional temperature of the reactorcontents; u : Input that corresponds to the adimensional flow rate of the heat-transfer fluid through thecooling sleeve, and y is the output of the system.For the purposes of this simulation the following values are used for the model’s constants:Da = 0,072; B = 8,0; β = 0,3; γ = 20,0Because Grey-Box Models are made up of two parts, one of which is the phenomenological modelrepresented by the different differential equations, and the other of which corresponds to the empiricalmodel represented by the neural network, the equations that represent the phenomenological model ofthe serial Grey-Box Model are the following:x′1 =−x1+0,072.(1− x1).ρ (4.4)x′2 =−x2+8.0,072.(1− x1).ρ +0,3.(u− x2) (4.5)y = x1 (4.6)Development of a Matlabr Toolbox for the Design of Grey-Box Neural Models 11Where ρ is the parameter that is difficult to obtain:ρ = e(x21+ x220)When equations 4.4 through 4.6 are discretized in the interval between t and t+1, the followingequations are obtained:x1(t+1) = x1(t)+(−x1(t)+0,072.(1− x1(t)).ρ).∆t (4.7)x2(t+1) = x2(t)+(−x2(t)+8.0,072.(1− x1(t)).ρ +0,3.(u− x2(t))).∆t (4.8)Thus the equations that constitute the phenomenological part of the serial Grey-Box Model, eitherwith OSA or MPO estimations, are the following:x1(t+1) = x1(t)+(−x1(t)+0,072.(1− x1(t)).ρ).∆t (4.9)x2(t+1) = x2t +(−x2(t)+8.0,072.(1− x1(t)).ρ +0,3.(u− x2(t))).∆t (4.10)y = x1(t+1) (4.11)4.2 OSA Estimation with 5% noiseThe validation obtained for the unknown parameter with 5% noise can be seen in Figure 3.Figure 3: Validation of the unknown parameter with OSA estimation and 5% noise.The validation obtained from the output of the system can be seen in Figure 4.Figure 4: Validation of the output of the system with OSA estimation and 5% noise.12 Gonzalo Acuña, Erika PintoThe error indices obtained from the serial Grey-Box Model with OSA estimation and 5% noise, forparameter ρ and the output of the system are presented in Table 4.1.Table 4.1. Error indices for parameter and the output of the system with an OSA estimation and 5%noise.Parameter ρ System OutputRSD 4.368098E-2 2.756748E-3RMS 1.733075E-2 1.944067E-2IA 9.111975E-1 9.846065E-1The error indices obtained are quite acceptable given that the acceptability values are RSD < 0.1,RMS < 0,1 and IA > 0.9 , as explained in Section 2.6 This indicates that the serial Grey-Box Modeldeveloped for the simulation of a CSTR process, with OSA estimation and 5% noise, is quite good.4.3 MPO Estimation with 5% noiseThe validation obtained for the unknown parameter with 5% noise can be seen in Figure 5.Figure 5: Validation of the unknown parameter with MPO estimation and 5% noise.The validation obtained from the output of the system can be seen in Figure 6.Figure 6: Validation of the output of the system with MPO estimation and 5% noise.The error indices obtained from the serial Grey-Box Model with MPO estimation and 5% noise, forparameter ρ and the output of the system are presented in Table 4.2.Table 4.2. Error indices for parameter ρ and the output of the system with an MPO estimation and 5%noise.Parameter ρ System OutputRSD 4.368098E-2 3.441815E-3RMS 1.733075E-2 2.427178E-2IA 9.111975E-1 9.764686E-1Development of a Matlabr Toolbox for the Design of Grey-Box Neural Models 13The error indices obtained are quite acceptable given that the acceptability values are RSD < 0.1 ,RMS < 0,1 and IA > 0.9. This indicates that the serial Grey-Box Model developed for the simulation ofa CSTR process, with MPO estimation and 5% noise, is quite good.5 ConclusionsGrey-Box Models constitute a real alternative for those real world processes for which the available apriori knowledge is incomplete, for example in a variety of industrial processes. As in Grey-Box Modelsonly some of the physical and/or chemical laws that represent the model are known, and there are un-known parameters that must somehow be estimated, multi-layered perceptron neural networks have beenemployed for their notable capacity to approximate complex functions on the basis of observed data.One of the advantages of this Toolbox, with reference to the creation and training of neural networks,is that neuron transfer functions for each of the hidden and output layers that make up the neural network,can be determined by the user, who can then create and train the neural network in a manner that is inkeeping with the specific problem that is to be modeled.Concerning the estimation of unknown system parameters, the developed Toolbox has the capacityto estimate as many parameters as necessary as long as the necessary data are available.The developed Toolbox (available at www.diinf.usach.cl/gacuna), allows the creation of serial Grey-Box Models, either with OSA (One Step Ahead) or MPO (Model Predictive Output) estimations, de-pending on what is to be modeled. With the creation of this Matlabr Toolbox, the end user has accessto a simple, trustworthy and fast tool which allows the development and subsequent manipulation ofdifferent serial Grey-Box Models, either with OSA or MPO estimations for solving a particular problem.This tool will become part of the Matlabr Toolboxes such that the user can access it at will simply bydownloading Matlabr. Thus this new tool becomes a real and simple alternative for modeling those realwork processes for which a priori knowledge available is incomplete.The error indices obtained are good, both for the unknown parameters and for each of the outputs ofthe system, even when the level of noise applied to the various input data was 5%, which indicates thatthe models developed are of good quality.AcknowledgementsThe authors wish to thank partial financing provided by Fondecyt Project 1040208.References[1] Psichogios, D.; Ungar, L. (1992). “A Hybrid Neural Network-First Principles Approach to ProcessModeling”, Computers & Chemical Engineering, 38(10): 1499-1511.[2] Thompson, M.; Kramer, M. (1994). “Modeling Chemical Processes Using Prior Knowledge andNeural Networks”, Computers & Chemical Engineering , 40(8):1328-1340.[3] Van Can, H; Hellinga, C.; Luyben, K.; Heijnen, J. (1996). “Strategy for Dynamic Process ModelingBased on Neural Network in Macroscopic Balances”, AIChE Journal, 42:3403-3418.[4] Thibault, J, Acuña, G., Pérez-Correa, R., Jorquera, H., Molin, P., Agosin, E., (2000) “A hybridrepresentation approach for modelling complex dynamic bioprocesses” Bioprocess Engineering,22(6):547-556.[5] Acuña G.; Cubillos, F.; Thibault, J.; Latrille, E. (1999). “Comparison of Methods for Training Grey-Box Neural Network Models”, Computers & Chemicals Engineering Supplement, 23:561-564.14 Gonzalo Acuña, Erika Pinto[6] Hornik K. Stinchcombe M., White H., (1989) “Multilayer Feedforward Networks Are UniversalApproximators”, Neural Networks, 2:359-366.[7] Billings S.A., Jamaluddin H. B. y Chen S., (1992) “Properties of Neural Network with Applicationsto Modeling Non-linear Dynamical System”, Int. J. Control, 55(1):193-224.[8] Pinto Armijo, Erika (2004), “A Matlab based application for developing grey-box models”, Memoriade Título de Ingeniería Civil Informática, Universidad de Santiago de Chile (in Spanish).[9] Hernández E.; Arkun, Y. (1992). “Study of the Control-Relevant Properties of BackpropagationNeural Network Models of Nonlinear Dynamical Systems”, Computers & Chemical Engineering,16(4):227-240.Gonzalo Acuña, Erika PintoUniversidad de Santiago de ChileDepartamento de Ingeniería InformáticaAvda. Ecuador No 3659 - Casilla 10233; Santiago, ChileE-mail: gacuna@usach.cl

Development of a Matlab(R) Toolbox for the Design of Grey-Box Neural Models

http://univagora.ro/jour/index.php/ijccc/article/view/2280/755

Development of a Matlab(R) Toolbox for the Design of Grey-Box Neural Models

Abstract

Similar works

Full text

Available Versions

Agora University Editing House: Journals