A computation method to smooth and differentiate data of the  z=/(x, y) kind is introduced. Requiring only that the datapoints be equi-distant in x and equi-distant in y, smoothing parameters can be calculated for general use. The greatest advantage of the method is that even higher-level mixed partial derivatives can be calculated directly from the datapoints

Kristyán, S.

English

Periodica Polytechnica (Budapest University of Technology and Economics)

A LEAST-SQUARE COMPUTATION METHOD FOR SMOOTInNG AND DIFFERENTIATION OF TWO-DIMENSIONAL DATA S. KruSTYAN Institute of Physics Labratory of Surface Physics, Technical University, Budapest, H-1521 Budapest Received October 21, 1987 Presented by Prof Dr. J. Giber Abstract A computation method to smooth and differentiate data of the z=/(x, y) kind is introduced. Requiring only that the datapoints be equi-distant in x and equi-distant in y, smoothing parameters can be calculated for general use. The greatest advantage of the method is that even higher-level mixed partial derivatives can be calculated directly from the datapoints. Introduction Random errors in experimental data have necessitated the development of mathe-matical and computation procedures for "filtering" the data; that is to say, removing a great part of the noise without losing any substantial amount of information. A widely-used mathematical method is that of the least-squares. [2, 3] Smoothing of the datapoints by considering only their 3-15 closest neighbors in the calculations, often gives the expected "filtering," but, in case of many hunderds of datapoints, the method is extremely slow and uneconomical. A simplified least-squares method, [4] which requires that the data be equi-dis-tant in each variable, uses "smoothing" and "differentiation" parameters, providing an easy-to-use algorithm with dramatically reduced computer time. Discussion For the one-dimensional case, a polynomial P(X) =ao +a1x +a2x2 + ... allx" is used, in a way, that the measured (Yi) and the calculated (P(Xi)) values differ the least: m Z (P(Xi)-Yi)2 = min (1) ;=1 (1) is then differentiated by and solved for the a/so After that, the value of the poly-nomial can be calculated in any point. 64 s. KRISTYAN If the basic system has equal distances in the abcissa (Ax=const., between any two neighboring datapoints), then the P(Xi) values, and the derivatives of the P(xYs are calculated directly from the measured Yi values. If we choose a coordinate system in which xi=i, and the distance between two neighboring points is 1, so the center of smoothing is Xo =0, its first left-side neighbor is X-I = -1, the third right-side neighbor is Xa =3, and if only the nearest m of the left-side neighbors and the nearest m of the right-side neighbors are considered in the calculations for each point, then we have i=m M ~ ( . I ·n )2 = L.J ao + al . 1 + ... Tan· 1 - Yi (2) i=-M which must be now minimized. (3) aM ~ ( . ."). 0 -a = L.J ao + al· 1 + ... + an· 1 - Yi 1 = a1 i=-III where we have n + 1 linear equations. Now, we arrange the J:ik's in a matrix [1], the a/s in a column-vector [A], and the J:Yi . ik'S in a column-vector [Y]: Furthermore, it is easy to see, that [1]· [A] = [Y] P(O) = ao dP(O) di (4) (5) For a given m, [1] is easily calculated and the system is solved for the a/so We find that k! ak = p(k)(O) = ..:.i=_-;:;.;.11I-::-:: __ N(k) (6) COMPUTATION FOR SMOOTHING AND DIFFERENTATION OF TWO-DIMENSIONAL DATA 65 In (6), the N's are the normalization constants, the S)O)'s are the smoothing para-meters, and the S)k)'S are the differentiation parameters. Since the S/s and the y/s are totally independent, the S/s will be the same if the degree of the polynomial and the value of 111 are unchanged. The greatest benefit (see (6)) is that the derivatives are calculated directly from the measured data_ When the parameters are used for smoothing only, we disregard the length of the interval (Lix), but for the rth derivatives (r= 1,2, ... ) the result must be multiplied by Lix-r. Our objective has been to develop a computation method, based on the same ideas, for two- and higher [5] dimensional data. The following polynomials could be used: Let us consider the two-dimensional case. First, the data must be equidistant in x(Lix=const.) and equi-distant in y(Liy=const.). The measured Zi values are put in a k X 1 matrix, where x changes along the rows and y changes along the columns. [1] In order to find the smoothing parameters, we use a coordinate system xi=i and Yj=j; the center of smoothing wiII always be the (0, 0) point. Similarly to the one-dimensional case, the 2111 nearest neighbors participate in the calculations, but now these are 2m neighbors in the x and 2m neighbors in the y directions; so, we shall use double summation. Putting i for x and j for y, and using the least-square method, we have 111 m ]v1= L: L: (ao+a1·i+ ... +as·rf+···-Zij)2 (7) i=-m j=-m Then, the minimization gives (8) m m L: L: (ao+ al· i+ ... + as· i'f+ ... -zij)i'f = ° i=-m j=-m m tU Now we put the L: L: i'jP values in matrix [K], the a values in column-vector i=-mj=-m [A] and another colunm-vector, [Z] will have the we can write [K] . [A] = [Z] 5 P. P. Electrical 33/1-2 111 m.a .b L: L: 1 ] • Zi' values. Then, .. J l=-m )=-m (9) 66 S. KRISTYAN The results for the a's are again related to the derivatives (now partial derivatives): m m Z Z S~~'p) 2,. " _ oP(O,O) _ i=-m j=-m I} IJ r.p.as - oi'ojP - -'-----"--,-N-:c;(~r,·p-):---- (10) where S\~ O)'s are smoothing, the S\j,b)'S are differentiation parameters (if a or b is nonzero), the N's are the normalization constants and Zij's are the values of the datapoints around Zo,o which is to be smoothed and/or differentiated. If the x-interval is .dx and the y-interval is .dy, then the result must be multiplied by .dx-r .dy-P for the mixed partial derivative, rth order in x and pth order in y. The results can be generalized to higher dimensions. [5]. Results and discussion Table 1 contains the parameters for one-dimensional smoothing, using third-degree polynomial, P=aO+alx+a2x2+asxS, with m=2. Table I Smoothing and Differentiation Parameters for One-Dimensional Data 111 Smooth dPjdx d2P/dx2 d3P/dx3 -2 -3 2 -1 -1 12 -8 -1 2 0 17 0 -2 0 1 12 8 -1 -2 2 -3 -1 2 N 35 12 7 2 Nis the normalization constant. To find the first derivative, for example, one has to multiply the second left-side value by 1, the first by - 8, the point itself by 0, the first right-side neighbor by 8 and the second by -1, and then the sum of the calcu-lated values is divided by 12 and divided by the stepsize (.dx). For the two-dimensional case the data wiII be in a data-matrix and the smoothing and the differentiation parameters are in a matrix, too. We have calculated the parameters for three different two-variable polynomials. Table Il contains the numbers for the P 2 = ao + a1 • x + a2 • Y smoothing curve. Table III for the PS=aO+al·x+a2·y+a3·x2+a4·xy+a5·y2, and Table IV for the P4=aO+a1· x+a2· y+as ' x 2+a4xy+a5. y2+ as ' xS+ai . x2y+asxy2+a9. y3. Depending on the data the user can choose the one that apparently best fits the measured values. For the sake of better understanding, we provide the mapping of the COMPUTATION FOR SMOOTHING AND DIFFERENTATION OF TWO-DIMENSIONAL DATA 67 parameters, in case of 111=2: (-2,2)(-1,2)(0,2)(1,2)(2,2) t (-2, 1)(-1, 1)(0, 1)(1, 1)(2, 1) Y (- 2,0)(-1,0) (0,0)(1,0)(2,0) (-2, -1)(-1, -1)(0, -1)(1, -1)(2, 1) (-2, -2)(-1, -2)(0, -2)(1, -2)(2, -2) x-The first number in the parentheses is the j index and the second is the i index. Table II Smooting and Differentiation Parameters Smoothing: N= 25 oP/ox: -2 -1 0 2 -2 -1 0 2 -2 -1 0 2 N= 50 -2 -1 0 2 -2 -1 0 2 '~-.---"""." .. ".".".".".""." .. "."."." oP/oy: 2 2 2 2 2 0 0 0 0 0 N= 50 -1 -1 -1 -1 -1 -2 -2 -2 -2 -2 The parameters have been tested with generated data. The data-matrices in the size of 60X48 contained the calculated values of Zl =8.0+273.0x2+5.0· xy2, Z2= =3.8+2.3x-4.5y, and z3=-7.7+5.5.x+11.3·y2, with a random error ranging from -10% to + 10% of the calculated values. As previously mentioned, the data-points are put in the data-matrix in a way that x changes along the rows and y changes along the columns. The stepsize for x was 0.1, for y it was 0.01. The calculations were so lengthy that even the generated values had to be calcu-lated by a Hewlett-Packard 85 personal computer. For Zl' the parameters from Table IV were used; for Z2 from Table H; and for Z3 from Table HI. The results proved to be excellent; the errors were reduced to 1/10-5* 68 S. KRISTyAN Table III Smoothing and Differentiation Parameters P3 = ao+atx+a2y+a3x2+a.lxy+asy2 Smoothing: -13 2 7 2 -13 2 17 22 17 2 7 22 27 22 7 N= 175 2 17 22 17 2 -13 2 7 2 -13 .~w~.w~ ••••• w •••••••••••••••••••••••••••• oP/ox: -2 -1 0 1 2 -2 -1 0 1 2 -2 -1 0 2 N= 50 -2 -1 0 2 -2 -1 0 2 . ______ o. _______ ....... ___________ •• __ ...• _. oP/oy: 2 2 2 2 2 1 0 0 0 0 0 N= 50 -1 -1 -1 -1 -1 -2 -2 -2 -2 -2 .- _____ 0 _______ .. _----------------------_. OP2/0X": 2 -1 -2 -1 2 2 -1 -2 -1 2 2 -1 -2 -1 2 N= 35 2 -1 -2 -1 2 2 -1 -2 -1 2 ---------------------------------------oP"/oy": 2 2 2 2 2 -1 -1 -1 -1 -1 -2 -2 -2 -2 -2 N= 35 -1 -1 -1 -J -1 2 2 2 2 2 .- __ -__ 0------------------------------_ .. oP"/oxoy: -4 -2 0 2 4 -2 -1 0 2 0 0 0 0 0 IV = 100 2 0 -1 -2 4 2 0 -2 -4 1/20 and even less fraction of the original. Further improvement could be made by using m=3 instead of 2. However, with m=2 416 peripherial datapoints are lost (these are used for the calculations but they themselves can not be treated because they do not have enough neighboring datapoints around them) with m=3 even more would have been lost (608). COMPUTATION FOR SMOOTHING AND DIFFERENTATION OF TWO·DIMENSIONAL DATA 69 Table IV Smoothing and Differentiation Parameters Smoothing: -13 2 7 2 -13 2 17 22 17 2 7 22 27 22 7 2 17 22 17 2 N= 175 -13 2 7 2 -13 ---.. -----_.- .. _------- .. _---_ ..... _- ......... oP/ox: 31 -44 0 44 -31 -5 -62 0 62 5 -17 68 0 68 17 -5 -62 0 62 5 N=420 31 -44 0 44 -31 ..... --_ ...... __ ... _-_ .. _- ... -................ oP/oy: -31 5 17 5 -31 44 62 68 62 44 0 0 0 0 0 N= 420 -44 -62 -68 -62 -44 31 -5 -17 -5 31 ....... _- ............ _- ....................... 2 -2 -1 2 2 -1 -2 -1 2 2 -1 -2 -1 2 N= 35 2 -1 -2 -1 2 2 -1 -2 -1 2 .. -_. __ .... __ .. -.----- ......... -_ ............. 2 2 2 2 2 -1 -1 -1 -1 -1 -2 -2 -2 -2 -2 N= 35 1 -1 -1 -1 -1 2 2 2 2 2 ................................... -.- ... __ ... 02P/oxoy: -4 -2 0 2 4 -2 -1 0 2 0 0 0 0 0 N= 100 .2 0 -1 2 4 2 0 -2 -4 ................. __ .................. __ ....... o"P/ox3 : -1 2 0 -2 -1 2 0 -2 2 0 -2 N=lO -1 2 0 -2 -1 2 0 -2 ............. -_ .............................. -1 -2 -2 -2 -2 -2 0 0 0 0 0 N=lO .2 2 2 2 2 -1 -1 -1 -1 -1 70 S. KRISTYAN Table IV continued [PPjax"ay: 4 -2 -4 -2 4 2 -1 -2 -1 2 0 0 0 0 0 N= 70 -2 1 2 1 -2 -4 2 4 2 -4 ._---------_ ...... ,,--------.......................... __ . -4 -2 0 2 4 2 0 -1 -2 4 2 0 -2 -4 N= 70 2 0 -1 -2 -4 -2 0 2 4 Conclusion Using the calculated parameters, the computations have proved to be extremely fast and very accurate. However, if the polynomials of the datapoints had not been known, there would have been the question, immediately: Which parameters to use? This is the case with experimental datapoints, too. The best way seems to be to start calculating the third-order partial derivatives with the parameters of Table IV, and if the results are all zeros (or very close to zero), but the second-order derivatives are not zeroes, then the data show a P3-behavior. If the second-order derivatives are zeros, too, then the data must be the P2 = ao + a1x + a2 • Y type. References 1. K"lORR, F. J. and HARRIS, J. M.: f".nal. Chem. 53, 272,1981. 2. GUEST, P. G.: "Numerical Methods of Curve Fitting", p. 349, University Press, Cambridge, England,1961. 3. CHEN"EY, W., KrNCAID, D.: '"Numerical Mathematics and Computing", p. 267, Brooks/Cole, Monterey, California, 1980. 4. SAVITZKY, A., GOLAY, M. T.: Anal. Chem. 36,1627,1964. 5. SZAMOSI, J., KRrsTYAN, S.: Yet unpublished calculations. Dr. Sandor KRISTYAN H-1521, Budapest 

A LEAST-SQUARE COMPUTATION METHOD FOR SMOOTHING AND DIFFERENTIATION OF TWO-DIMENSIONAL DATA

https://pp.bme.hu/ee/article/download/4584/3689

A LEAST-SQUARE COMPUTATION METHOD FOR SMOOTHING AND DIFFERENTIATION OF TWO-DIMENSIONAL DATA

Abstract

Similar works

Full text

Available Versions

Periodica Polytechnica (Budapest University of Technology and Economics)