Neural Networks were used to build a model that would predict the   transit times and the levels of trace amounts of radioactive   components at the outlet of a sewage treatment plant.  The Neural   Networks are compared with more conventional approximation methods as   a tool for prediction

Airey, P.

Barry, J.

Crawford, J.

English

Australian Mathematical Society (AustMS): E-Journals

ANZIAM J. 42 (E) ppC200{C216, 2000 C200Predicting transit times and flowconcentrations of contaminants through ametropolitan sewage systemJ. Barry J. Crawford P. Airey (Received 7 August 2000)AbstractNeural Networks were used to build a model that would predict thetransit times and the levels of trace amounts of radioactive componentsat the outlet of a sewage treatment plant. The Neural Networks are∗Australian Nuclear Science and Technology Organisation Private Mail Bag 1,Menai 2234, Australia. mailto:jmb@ansto.gov.au, mailto:jc@ansto.gov.au,mailto:pla@ansto.gov.au0See http://anziamj.austms.org.au/V42/CTAC99/Barr for this article and ancillaryservices, c© Austral. Mathematical Soc. 2000. Published 27 Nov 2000.Contents C201compared with more conventional approximation methods as a toolfor prediction.Contents1 Introduction C2022 Physical system C2023 Conventional model C2043.1 Mathematical basis function . . . . . . . . . . . . . . . . . C2063.2 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . C2074 Neural network model C2084.1 ANN model formulation . . . . . . . . . . . . . . . . . . . C2104.2 Stability of the ANN model . . . . . . . . . . . . . . . . . C2115 Evaluation of the models C212References C2151 Introduction C2021 IntroductionThe third author challenged the others to examine the potential of using Neu-ral Networks to model complex dynamic behaviour in environmental systems.The motivation was to experiment with a black box technique instead of otherphysical modelling systems which were considered too costly to develop andimplement. The following problem was considered suciently complex to beof use in evaluating the value of Neural Networks in modelling such systems.ansto routinely discharges low levels of radionuclides, including tritiumto the Cronulla sewage system under the appropriate authorisation. Underits Health, Safety and Environment Policy, the Organisation is committedto undertaking its activities consistent with national and international bestpractice. This involves a commitment to continuous improvement.To eect continuous improvement, it is important for ansto to be ina position to predict the transit time of the discharge before it reaches thetreatment plant and the levels of radioactivity as it passes through the plantunder a wide range of conditions.2 Physical systemThe sewage system to which ansto is linked is dicult to model as it con-sists of many conduits, holding locations and intermediate pumping stations2 Physical system C203of diering rates and delay times. In addition to normal household and indus-trial discharges there are many other environmental and operational factorsthat are dicult to account for in a conventional modelling approach. Thecomplexities of the system were not incorporated directly in the models.Tritium (as hto) was modelled in this study as it behaves as water and isnot retarded by adsorption on the pipeline walls or by the sewage sludge. Thedischarges from the ansto holding tanks are made for periods of between 3to 7 hours. The tritium concentration is known and constant due to a mixingprocess in the tanks prior to discharge. The euent is discharged into thesewage system at a reasonably constant rate. In the models it is representedby hourly varying pulses. The ansto contribution to the system is approx-imately 4% of the water volume carried by the sewage system. Samples atthe Cronulla Sewage Treatment Plant (cstp) are taken at hourly intervals,starting before the euent reaches the plant and the measurements are con-tinued until the material has passed through. The samples are returned tothe laboratory for tritium assay. Because of possible losses from the systemdue to damaged pipes and possible retention of portion of the release in in-termediate stations, the system is not necessarily conservative. Consequentlyno attempt was made to force conservation in the models.Two empirical modelling approaches were adopted. A neural networkwas developed in a typical black box approach to predict the flow times andconcentrations so as to avoid the need for nding a suitable mathematicalapproximation function. The network was trained with data recorded froma limited number of releases from ansto. In a more traditional response3 Conventional model C204model, an approximation function was developed and parametrised throughminimisation of least squares error component. As in the case of the neuralnetwork, specic details of the sewage network was not included in the model.3 Conventional modelWith the conventional modelling approach, it is advisable to consider possiblemathematical mechanisms that may describe the physical system. If the timevarying tritium load from the ansto facility is represented by I(t) and thetritium load measured at the cstp by Cout(t), then an integral equation ofthe formCout(t) =Z t0I(s)f(t; s))ds (1)may be used to relate the two quantities provided the kernel function f(t; s)can be determined. We persist with a kernel function of this form, however,the situation may well be more complex for f(s; t; I(s)) may turn out tobe more appropriate form. Equation 1 is a Fredholm Integral equation ofthe rst kind, where normally the Cout(t) and f(s; t) are known and I(t) issought. These problems can be dicult to solve because they are typicallyill-posed [2]. The ill-posed nature of the problem shows up in the recovery ofthe kernel functions because the kernel does not always depend continuouslyon the data.3 Conventional model C205If one fails to recognise the integral equation formulation either in settingup a neural network solution or in a standard least squares approximation,then there can be important implications for the solutions obtained.One may set up an approximation to the physical system or attempt todiscretise the integral equation throughCout(t) =nXi=0I(ti)gi(t− tdelay; ai); (2)where the inputs are treated as n + 1 pulses at times ti generally on anhourly basis, when the discharge flow rates were measured and recorded. Theoutput loads are related to the input loads through the convolution functiongi(t− tdelay; ai), where tdelay is the time of flight for a particular stage of therelease. In the present model, tdelay is assumed constant for every stage of aparticular release. The functional form is parametrised by sets of parametersai.As it stands, there are a set of n + 1 convolution functions to be deter-mined for each release used to determine the model. This would require thecomputation of a set of parameters ai for each release. As part of the simpli-cation it is assumed that the same parameters apply for every release, thiscorresponds with the decision not to use the f(s; t; I(s)) form of the kernel.3 Conventional model C2063.1 Mathematical basis functionThe convolution function g(t; ai) of equation (2), is dened with linear hatbasis functions ig(t− tdelay) =rXj=1ajj(t− tdelay); (3)based on r nodal points in time T1 < T2 < : : : < Tr. The points are chosenat r suitable time displacements from t−tdelay . Using t′ = t−tdelay , the basisfunctions i are dened:1 =t′ − t0T1 − t0 for t′< T1; (4)j =t′ − Tj−1Tj − Tj−1 for Tj−1  t′  Tj j = 2; 3; : : : ; r, (5)j =Tj+1 − t′Tj+1 − Tj for Tj  t′  Tj+1, j = 1; 2; : : : ; r − 1, (6)= 0 elsewhere; (7)r = 1− t′ − Trt1 − Tr ; for t′> Tr; (8)where t0 is the time at which the release of euent from ansto commencesand t1 is the time after the eect of any discharge passes through the system.3 Conventional model C207In parametrising the model all releases are considered simultaneously andthe parameters are recovered by minimisingpXk=1mkXj=1"Ckout−observed(tkj )−Pnki=1Prl=1 Ik(tki )all(tkj − tkdelay)j;k#2; (9)where p dierent releases are used; for the kth release there are mk obser-vations at the outfall, nk input observations and a separate delay constant,tkdelay for each of the p releases. Variance estimate j;k were not available, sounit variance was assumed. Minimisation was easily obtained with a quasi-Newton algorithm. When the model is used in a predictive mode, tkdelay isassumed to be linearly related to the time when the release originated.3.2 StabilityThe model was rst parametrised using r = 7 nodal points. The model wasable to reproduce the observed loads at the cstp and the delay times ratherwell. However, a sensitivity test on the parameters was undertaken and asanticipated, large fluctuations in the recovered parameters were detected forsmall perturbations of the underpinning data; as typically experienced inill-posed problems of this form. In a move to reduce these fluctuations, twoapproaches were adopted. Tikhonov regularisation[2] was implemented andalternatively, the number of nodal points allowed in the basis functions wererestricted. The second proved eective and for its inherent simplicity was4 Neural network model C208the method ultimately adopted. To evaluate the value of the model further,the sensitivity of each parameter to data error was tested by Monte Carlosimulations for r = 3; 5 and 7. A 1:0% perturbation, randomly normallydistributed was applied to the input data. There were four releases used inthe parametrisation, hence four values for tdelay were recovered as well as thespatial parameters a. The standard deviation of each parameter was esti-mated in the Monte Carlo simulation and each is expressed as a percentageof the recovered parameter in Table 1, where an ‘x’ in the table indicates theoptimisation algorithm consistently drove that recovered parameter to zero.For r > 3 the standard deviations of the parameters are so large as to bringinto doubt the reliability of the model, even though the model predictionsmatch well the observed loads.4 Neural network modelWhile the Articial Neural Networks (anns) [1], [4] approach is similar to theconventional optimisation approach (in that an empirical model is derivedbased on the data acquired from past experience) the advantage is that onedoes not need to dene the form of the function. Two approaches wereconsidered in representing the problem using anns. In both approaches theoutputs of the ann are the measured hourly concentrations at the outfallfrom the treatment plant. Using this approach, the hourly concentrations at4 Neural network model C209Table 1: Standard deviation of each convolution function parameter, ex-pressed as a percentage of the parameter.parameter r = 3 r = 5 r = 7a1 0.3 10. 19.a2 0. 272. 322.a3 3.2 5.7 88.a4 x 3.1a5 21.9 xa6 xa7 31.tdelay1 0.00 4.2 0.03tdelay2 0.08 1.5 18.tdelay3 0.06 0.00 .006tdelay4 0.09 6.1 6.4 Neural network model C210the cstp, Cout(tj), for a given release can be represented as follows:Cout(tj) = ftj (I; W ); (10)where ftj are non-linear mapping functions, W is a vector of adjustableparameters or weights, and I is a vector describing the release into the system.The measured hourly discharge concentrations form the vector I in therst approach while parameters describing the discharge concentrations formI in the second approach. The rst approach will not be discussed here, as itwas considered restrictive, in that the duration of a discharge and the timeof day when this occurred were xed. Increasing the operational hours ofdischarges would result in a much larger network and hence a requirement ofa larger training set of observations. In this rst approach, while the observedmeasurements were able to be learned and reproduced fairly accurately, thepredictive nature of the network was inferior. This is not surprising given thesize of the network (a 8{25 network, with 200 weights to be recovered) andlimited amount of training data available. The second approach is describedin subsequent sections.4.1 ANN model formulationThe ann model is developed assuming a constant discharge rate and concen-tration. The inputs to the ann (components of vector I) are the start timeof the release, the duration of the release, the tritium load of the discharge4 Neural network model C211and the water discharge rate into the system. The outputs of the ann arethe concentrations at the outfall from the treatment plant.A feed-forward ann, using supervised learning [5] was considered appro-priate for this problem and a package [6] was obtained for use. The packageoers a number of activation functions and learning algorithms. A Gaussianactivation function was chosen and the quickprop [3] learning algorithm.In choosing a suitable ann architecture, a number of experiments werecarried out with hidden layers, however, it was found that adding a hiddenlayer did not improve the performance. In most cases, the predictive ca-pability of the network was improved when the hidden layer was removed;hence a network without hidden layers was considered. To reduce the num-ber of weights to be recovered, the measurements at the outfall were limitedbetween 6 and 21 hours after the release, resulting in a 4{16 network.4.2 Stability of the ANN modelTo evaluate the value of the neural network model, a number of studies werecarried out as described in [5], e.g. limiting the magnitude of the weights andensuring that the distribution of weights formed a centred histogram. Theseconditions were satised, however they are not reported.In addition, for this comparison, a sensitivity test on the weights wasundertaken as described in Section 3.2, i.e. a 1:0% random normal perturba-5 Evaluation of the models C212tion on the input was applied. The standard deviation of each weight wasestimated using Monte Carlo simulation. The standard deviation of eachweight, expressed as a percentage of the recovered weight, varied from 0:1%to 20:0%, with four outliers of 27%, 63%, 40% and 23%. The larger standarddeviations corresponded to the sections of the curves with sharp rise and fall.5 Evaluation of the modelsTo parametrise both models, data from four releases were used, these werethe rst four of six sets of ansto data recorded in chronological order. Theremaining two were retained to test the predictive worth of the two models.The conventional and neural network models obtained for a typical trainingset are shown in Figures 1 and 2, where the observed and predicted loads aredisplayed. Superimposed on the predicted loads is an estimate of the stan-dard deviation obtained through a Monte Carlo simulation, where a randomnormally distributed perturbation of 10:0% of the release load was applied.In each case the predictions produced are considered useful. The models givea reasonable estimate of the time of arrival of the euent at Potter Point,the level of the load and the time it takes to pass through the plant.The predictive power of each approach is shown for the rst test releasein Figures 3 and 4. In this test case there were signicant dierences in thedischarge regime to those used in training the models. Consequently themodels are being tested in a truly predictive mode. The more conventional5 Evaluation of the models C213Figure 1: Observed and predicted loads for a release used in developing theconventional model, standard deviations are shown on prediction.Figure 2: Observed and predicted loads for a release used in developing theann model, standard deviations are shown on prediction.5 Evaluation of the models C214Figure 3: Observed and predicted loads for a release used in testing theprediction of the conventional model, standard deviations are shown on pre-diction.model is a superior predictive tool in this instance, due no doubt to theregularisation forced through the reduced kernel function.In a second test case, the volume discharged from ansto were consider-ably greater, however, the results are not shown. Unfortunately in this case,while the predicted arrival time is satisfactory, the model predicts a shorterthan actual passage through the plant. This is oset with a correspondinglyhigher predicted load in the plant for the shorter period, which brings intoquestion the use of empirically derived predictive models for situations thathave marked dierences to those for which they are parametrised. Such aresult is not surprising where the training data is limited. An importantissue is raised which cannot be answered until more release data is available;References C215Figure 4: Observed and predicted loads for a release used in testing theprediction of the ann model, standard deviations are shown on prediction.is it possible to do better with the current empirical models with additionaltraining data, or do the models require revision to accommodatechanged cir-cumstances? In the sewage system there are countless extraneous factorswhich have not been accommodated in either approach. Their absence was adriving factor behind the use of neural networks, however, the ann has notproven to be superior in this study.References[1] C.M. Bishop. Neural Networks for Pattern Recognition. ClarendonPress, Oxford, 1997. C208References C216[2] H.W. Engl, M. Hanke, and A. Neubauer. Regularisation of InverseProblems. Kluwer Academic, Dordrecht, Boston and London, 1996.C204, C207[3] S.E. Fahlman. Faster-learning variations on back-propagation: anempirical study. In D. Toretzky, G.E. Hinton, and T.J. Sejnowski,editors, Proceedings of the 1988 Connectionist Models Summer School,pages 38{51, San Mateo, CA, Morgan Kaufmann. C211[4] W.S. Sarle. FAQ, available fromftp://ftp.sas.com/pub/neural/FAQ.htm . C208[5] K.S. Swingler. Applying Neural Networks. Academic Press, London,1996. C211, C211[6] D.R. Tveter. Professional Version Basis of AI Backprop,mailto:drt@christianliving.net . C211

Predicting transit times and flow concentrations of contaminants through a metropolitan sewage system

https://journal.austms.org.au/ojs/index.php/ANZIAMJ/article/download/596/464

Predicting transit times and flow concentrations of contaminants through a metropolitan sewage system

Abstract

Similar works

Full text

Available Versions

Australian Mathematical Society (AustMS): E-Journals