The ERS-1 satellite carries a scatterometer which measures the amount of radiation scattered back toward the satellite by the ocean's surface. These measurements can be used to infer wind vectors. The implementation of a neural network based forward model which maps wind vectors to radar backscatter is addressed. Input noise cannot be neglected. To account for this noise, a Bayesian framework is adopted. However, Markov Chain Monte Carlo sampling is too computationally expensive. Instead, gradient information is used with a non-linear optimisation algorithm to find the maximum em a posteriori probability values of the unknown variables. The resulting models are shown to compare well with the current operational model when visualised in the target space

Cornford, Dan

Nabney, Ian T.

Ramage, Guillaume

English

Aston Publications Explorer

Neural Computing Research GroupDept of Computer Science & Applied MathematicsAston UniversityBirmingham B4 7ETUnited KingdomTel: +44 (0)121 333 4631Fax: +44 (0)121 333 4586http://www.ncrg.aston.ac.uk/A Neural Network Sensor Modelwith Input NoiseDan Cornford and Guillaume Ramage and Ian T. Nabneyd.cornford@aston.ac.ukTechnical Report NCRG/98/021 October 22, 1998AbstractThe ERS-1 satellite carries a scatterometer which measures the amount of radiation scattered backtoward the satellite by the ocean's surface. These measurements can be used to infer wind vectors.The implementation of a neural network based forward model which maps wind vectors to radarbackscatter is addressed. Input noise cannot be neglected. To account for this noise, a Bayesianframework is adopted. However, Markov Chain Monte Carlo sampling is too computationallyexpensive. Instead, gradient information is used with a non-linear optimisation algorithm to ndthe maximum a posteriori probability values of the unknown variables. The resulting models areshown to compare well with the current operational model when visualised in the target space.Keywords: Non-linear regression; input uncertainty; wind retrieval; scatterometer2 A Neural Network Sensor Modelwith Input Noise1 IntroductionThe ERS-1 satellite was launched in 1991 by the European Space Agency. It carries a scatterometerwhich measures the return radar power from three antennae that form a swathe to the right side ofthe satellite ground track. Some necessary technical background and notation are given in (Nabneyet al., 1998, this issue). As the three antennae sweep a 500 km wide swathe, the incidence angle,, with which the cells are illuminated varies.In order to infer a wind eld from scatterometer measurements, we need a probabilistic forwardmodel for P (oju; v; ), where ois the backscatter triplet and (u; v) are the wind vector compo-nents. Several algorithms for wind retrieval use deterministic forward models, such as the empiricalmodel, CMOD4 (Oler, 1994), used operationally. CMOD4 assumes the 3 antennae are equivalentand has a functional form dened by:olin= B0(1 +B1cos() +B3tanh(B2) cos(2))1:6(1)where B0, B1, B2, B3are complicated functions of the wind speed, s, and the beam incidence angle,.  is the wind direction relative to the beam look angle and olinis the backscatter measuredon a linear scale (Stoelen and Anderson, 1997). A forward model based on neural networks ispresented in (Mejia et al., 1998), but, it ts the observations poorly in ospace (Ramage, 1998)as it was trained without accounting for input noise.2 Input uncertainty2.1 Evidence for input uncertaintyIn this problem the targets, o, are multi-dimensional and they can be plotted in 3D space whereois the logarithm of olin, rendering the noise distribution additive. Using such a representation,we show that input uncertainty cannot be neglected with respect to noise in the target variables.In Fig. 1(a), a large number (10,000) of otriplets are represented in target space, for all windspeeds and directions, and for a xed incidence angle,  = 34:9. These points are projected on theplane o= 0 for the mid beam. As the measurements depend predominantly on two geophysicalvariables (wind speed and direction), they lie on a well dened manifold. Their distance from themanifold is small (around 0.2 dB) and is mostly due to instrumental noise.These satellite measurements are labelled with wind vectors obtained from Numerical WeatherPrediction (NWP) models. In Fig. 1(b), the measurements are selected so that the speed, givenby the NWP model, lies in the range 9  s  10 ms 1. The solid surface corresponds to theoperational forward model, CMOD4, plotted over the same range of wind speeds. This surfaceshows where the points should lie in the absence of input noise. The spread of the points is verylarge (around 5 dB) compared to instrumental noise. Therefore, input noise cannot be neglected.2.2 Eects of input uncertaintyThe test error of a dataset with noisy inputs is a poor measure of model accuracy. Using anindependent test set is the most common way for assessing the quality of a regression model(Bishop, 1995) in the absence of input noise. If the input noise is not accounted for during trainingof a nonlinear model, model bias is likely to result (Wright, 1998). Thus, the computed test errorA Neural Network Sensor Modelwith Input Noise 3−25 −20 −15 −10−26−24−22−20−18−16−14−12−10−8σo fore−beam (dB)σo aft−beam (dB)Figure 1: The manifold in ospace at an incidence angle of 34:9. The points on the right plotare sub-sampled from the points on the left plot with the criterion s 2 [0; 9 ms 1].The surface is drawn for the same range of speeds. For reference, the other linesrepresent wind speeds of 6 (bottom left) and 13 ms 1(top right).is signicantly aected by input noise and is not a true measure of model accuracy. Such a testerror only tells us how good the model is at predicting targets from noisy inputs, but we want toinfer the regression over noiseless inputs.As explained above, the targets should lie on a manifold, for a xed incidence angle. The surfacedened by a model trained without accounting for input noise does not t this manifold (seeSection 6). Thus, one fundamental test for a model is its ability to t this theoretical surface.In order to improve the accuracy of retrieval of high wind speeds, which are of most interest tometeorologists, the training dataset is sub-sampled from the available data.2.3 Modelling the noiseIn order to train a model while accounting for input noise, this noise must be modelled. Thenoise is dicult to describe in terms of wind speed and direction (Stoelen and Anderson, 1997).Indeed, the noise in the speed component has a complicated skewed distribution at low windspeeds. However, in terms of Cartesian wind components, the noise distribution can be describedby a spherical Gaussian distribution (Stoelen and Anderson, 1997). The noise in target space isalso assumed to have a spherical Gaussian distribution with a much smaller variance. The noisevariances are set using results from (Stoelen and Anderson, 1997).3 Bayesian learning frameworkA Bayesian approach to neural network modelling with input uncertainty is proposed in (Wright,1998). The posterior probability of a new target is:P (tj x; D0) /ZwP (tj x;w) P (w jD0) dw; (2)4 A Neural Network Sensor Modelwith Input NoisewhereD0is the noisy training set, tis the target and P (tjx;w) is the target density conditionalon a noiseless input xmodelled by a neural network with weights w. P (w jD0) can be expanded:P (tj x; D0) /ZwP (tj x;w)ZxnYnP (tnj xn;w)| {z }P1P (znj xn)| {z }P2P (xn)| {z }P3P (w)| {z }P4dxndw;(3)where tnare the targets in the training data, xnare the corresponding noiseless (latent) inputs andznare the associated noisy inputs. Training the network consists of determining P (w jD0). Thiscan be done using Markov Chain Monte Carlo sampling. From (3), it can be seen that the Markovchain samples from fxn;wg. Thus the size of the data set has to be reduced as much as possible inorder to keep the Markov chain at a reasonable size. However, to ensure a highly accurate modelfor all possible inputs a large data set is required. A practical alternative to sampling is to computethe required derivatives and determine the maximum a posteriori probability values of fxn;wgusing, for instance, a scaled conjugate gradient algorithm. This is the approach we adopt.4 Chosen architectureθ~E1σ~sin(  )χcos(  )χcos(2  )χθangleincidence sMLPNeuralNetworkovaluesimulatedtrainingvalueχwσooσsdirectionwindrelatives~wind speedFigure 2: Architecture of the neural network model. Boxed variables are determined by non-linear optimisation.Although we need P (oj u; v; ), wind vectors should not be presented to the network as vectorcomponents. At constant speed and xed incidence angle, ovaries roughly as cos(2) (see (Mejiaet al., 1998)). Thus cos(2) forms an input to the network. Both cos() and sin() are also usedas inputs to respect the continuity of . All boxed variables in Fig. 2 are variables which will beoptimised. All variables are normalised to zero mean with a common standard deviation around0.5{0.7. As normalisation functions are part of the model, we want to keep them simple. The windspeed is related to the normalised wind speed by an exponential function to ensure it is positive.5 Practical implementationIn order to compute the maximum a posteriori of P (w j D0), we compute the four errors Ei=  ln(Pi), see equation (3). These terms are:A Neural Network Sensor Modelwith Input Noise 5 E1=   ln(QnP (oj ss; s; ;w)) is the error of the model, calculated for the observedsatellite measurements and for sampled wind vectors (ss; s) which tend to the noise freevalues during training. E2=   ln(QnP (ss; sj s; )) is the error due to the sampled wind vectors diering fromtheir associated noisy wind vectors. E3=   ln(P (s; )) =   ln(P (s)) is the prior distribution of true wind speeds in the trainingset. This is uniform in relative direction and so depends only on speed. E4=   ln(P (w)) is the prior over the weights which regularises the neural network (Bishop,1995).P1is assumed to be spherically Gaussian in target space, thus:E1=X(os  o)2=(22o); (4)where the sum is over the three ovalues and the patterns in the training set, ois the standarddeviation of the errors in the omeasurements and osis the output obtained propagating thesampled inputs (ss; s) through a Multi-Layer Perceptron (MLP) with M hidden units (Fig. 2).This can be written:~o=MXj;k=1wk;jtanh(wj;1~ + wj;2sin() + wj;3cos() +wj;4cos(2) + wj;5~s+ wj;0) + wk;0;(5)where~ denotes the associated normalised quantities. The output is then transformed into real ospace by inverting the normalisation.P2is assumed to be spherically Gaussian in vector components of the wind with standard deviationu, so that:E1=X(us  u)2+ (vs  v)2=(22u): (6)The wind speed distribution, P3, is represented by a uniform distribution between 4 and 28 ms 1,similar to that in the dataset, with smooth Gaussian decrease at the ends:E3 = (4  ss)2=(22u)E3 = (28  ss)2=(22u)ss<4 ms 1ss>28 ms 1(7)Finally, P4corresponds to the weight decay prior:E4=Xww2=(22w): (8)As noted previously, in order to compute the integral (2) using Monte Carlo integration, we need tobuild an independent set, fs; ;wg, drawn from the expansion of P (wjD0) in (3). We cannot obtainthis in a reasonable time so a non-linear optimisation is performed using gradient information. Thefollowing derivatives are computed analytically:@Ei@ ~;@Ei@~s;@Ei@w; i = 1:::4 (9)A training set of 10,000 patterns is used and thus we compute more than 20,000 derivatives ateach step in the optimisation. In conventional training of neural networks we only need @E1=@w.6 A Neural Network Sensor Modelwith Input NoiseFigure 3: In target space, the surface dened by the model lies inside the target points (left plot)when we t a model ignoring input noise. This is corrected if we account for inputnoise (right plot). Some parts of the manifold are removed to allow visualisation.Shading represents wind direction, lines on the surfaces represent constant windspeeds of 4, 8, 12, 16, 20 and 24 ms 1. The surface is not drawn for 12{16 ms 1.6 Results and DiscussionAssessing the quality of a model is dicult; we use graphical representations in this study. Fig.3shows the neural network based model trained without (left) and with (right) accounting for inputnoise. The surface dened by the model accounting for input noise ts the target data well in ospace, while the model trained without accounting for input noise lies largely within the interiorof the manifold dened by the observations.The model trained accounting for input noise can be seen to t the observed ovalues poorly atlow wind speeds. This is believed to be a result of data selection. All noisy wind speeds below4 ms 1are discarded1. However, omeasurements corresponding to true wind speeds below4 ms 1are still present, but all of them are labelled with over-estimated speeds above 4 ms 1.This sub-sampling on the basis of noisy data introduces bias in the data (Ramage, 1998) whichwill be removed by careful data selection in ospace in future work.The models presented here are dierent from CMOD4 (Fig. 4 (left)) which has a strongly restrictivefunctional form, and ts the observations poorly at both high and low wind speeds.In Fig. 4 (right) we can see the evolution of the wind vectors during optimisation of the modelaccounting for input noise (2w= 10; 2u= 1:5 ms 1; 2o= 0:2 dB, 2500 iterations of the scaledconjugate gradient algorithm). The lines appear organised, which means the vector adjustmentsare correlated. This should not happen for good models as the wind vectors in the training set areselected so that their errors are uncorrelated.The change in direction is slightly direction dependent. Absolute wind direction has no simplegeophysical meaning as it is relative to each satellite beam. Therefore, this dependency is probablydue to some mist in the model, rather than systematic errors in the NWP wind directions. Thechange in speed appears speed dependent, and is due to the uniform selection of data from a xedspeed range as noted earlier. Finally, the change in speed is larger than the change in direction.1At wind speeds below 4 ms 1the omeasurements become unreliable (Oler, 1994).A Neural Network Sensor Modelwith Input Noise 7−30 −20 −10 0 10 20 30−30−20−100102030u (ms−1)v (ms−1)Figure 4: Left plot: The currently operational model, CMOD4, plotted as in Fig. 3. Rightplot: crosses represent NWP winds (used as starting points). Straight lines link themto wind vectors at the end of optimisation. All incidence angles are represented andthe number of points is reduced for clarity.Describing the noise on the winds in terms of (; s) components might improve the model, althoughthis eect may be due to the shape of the manifold in ospace.7 ConclusionsThe design of a non-linear regression model with input uncertainty is a dicult task, especiallywith multidimensional inputs and outputs. Even if the neural network has enough degrees offreedom to model the mapping, noise on the variables should be modelled accurately to obtaingood results. The method we propose could be enhanced to determine the noise variances 2u(or2, 2s) and 2oas part of the modelling procedure by introducing a prior over these variances.Due to the large number of derivatives that must be computed, training takes roughly twice as longas conventional training of neural networks in our implementation. Unlike a fully Bayesian treat-ment, only the maximum a posteriori probability value of w is computed so forward-propagationthrough the network is fast when we have noise free inputs. If we have noisy inputs then it isnecessary to consider an additional integral over the inputs, to retrieve the output given a noisyinput (Wright, 1998).A proper data selection, based on sub-sampling using target (o) information, should minimise theerrors associated with data selection based on noisy variables. Future work will also investigatethe retrieval of wind elds using the forward model (Nabney et al., 1998). This will enable aquantitative comparison of models.AcknowledgementsThis work is supported by the European Union funded NEUROSAT programme (grant numberENV4 CT96-0314). We thank Andy Wright for useful discussions.8 A Neural Network Sensor Modelwith Input NoiseReferencesBishop, C. M. 1995. Neural Networks for Pattern Recognition. Oxford, UK: Oxford UniversityPress.Mejia, C., S. Thiria, N. Tran, M. Crepon, and F. Badran 1998. Determination of the GeophysicalModel Function of the ERS-1 Scatterometer by the Use of Neural Networks. Journal ofGeophysical Research 103, 12853{12866.Nabney, I. T., D. Cornford, and C. K. I. Williams 1998. Bayesian Inference for Wind FieldRetrieval. Neurocomputing Letters . submitted.Oler, D. 1994. The Calibration of ERS-1 Satellite Scatterometer Winds. Journal of Atmo-spheric and Oceanic Technology 11, 1002{1017.Ramage, G. 1998, September. Neural Networks for Modelling Wind Vectors. Msc thesis, AstonUniversity.Stoelen, A. and D. Anderson 1997. Scatterometer Data Interpretation: Estimation and Vali-dation of the Transfer Function CMOD4. Journal of Geophysical Research 102, 5767{5780.Wright, W. A. 1998. Neural Network Regression With Input Uncertainty. In Neural Networksfor signal processing, Volume 8, pp. 284{293.

A scatterometer neural network sensor model with input noise

https://publications.aston.ac.uk/id/eprint/1411/1/NCRG_98_021.pdf

A scatterometer neural network sensor model with input noise

Abstract

Similar works

Full text

Available Versions

Aston Publications Explorer