KÓNYA, L.

English

Periodica Polytechnica (Budapest University of Technology and Economics)

DISCRETE ORTHOGONAL FUNCTIONS APPLIED TO FILTERING AND DATA COMPRESSION By L. K6NYA Department of Automation, Technical University, Budapest Received December 22,1976 Presented by Prof. Dr. F. CsAK1 Discrete orthogonal transformations provide a useful tool of computer aided signal processing. In signal processing an important problem is the separation of the signal from the noise. Another problem is the efficient storage of the signal in the computer memory. The paper gives a short review of this field. Mathematical bases A function g(t) may be expanded into a series of orthogonal functions [4]: g(t)= L: aJ;(t) i=j For the orthogonal system offunctions {Ht)}: I U;,~;, ia ~ J f;(t) ·jj(t) dt=W o if i-,= j if i=j The coefficients {aJ are determined by the relationship [4]: , ai=~ J flt)g(t) dt i=1,2, ... o where I is the range of orthogonality. (1) (2) (3) Such an expansion is e.g. the Fourier series, where the orthogonal functions are a set of trigonometric functions, and the orthogonality range is (0, 2n) [5]. The above equations may be written in discrete matrix form by dividing (0, I) into N=2n(n>0, integer) subintervals, and within each "interval J;(t) and g(t) are supposed to be constant. 24 L.KONYA Introducing the designation k= tiN, we obtain the relationships and N g(k)= 2: aJi(k) i=1 1 N a i=- 2: g(k) -i;(k) N k=1 k=1,2, .. . ,N i= 1, 2, ... , N which may be written, introducing the symbols in the form of gT=[g(I), g(2), .. " g(N)] fl=[fi(I),J;(2), ., .,J;(.N)] i= 1,2, ... , N 1 a=-FO" N to' (5) (6) (6a) (7) This shows that the connection between the sampled functions and the ex-pansion coefficients is given by an orthogonal transformation, which may be rep-resented by a matrix multiplication. For the matri.x F it is true that where I is the unity matrix where T is the symbol of transposition. (8) The rows and columns of Fare orthogonal to each other which is the presentation of (2) in discrete form. What is the apparent meaning of the transformation? The original vector g formed from the sampled function values is a vector of an N-dimen-sional space. Upon orthogonal transformation it becomes a vector of another space characterized by the transformation matrix [4]. Performing the matrix mUltiplication required for the transformation may be accelerated by applying the fast transformation technique developed for several dis-'" crete systems of orthogonal functions, e.g. Fast Fourier Transformation (FFT), Walsh-Hadamard Transformation (WHT), Haar Transformation (HT), Discrete Cosine Transformation (DCT) [1]. Fast transformations significantly reduce the time and space requirement. ISCRETE ORTHOGO,YAL FUNCTIONS 25 The generalized Wiener filtering The mathematical model is shown in Fig. 1. The input is the vector z, sum of signal x and noise w; F designates the orthog-onal transformation, A is the filter matrix, and x is the filtered vector. The problem is to find A:so that x approximates x best. Let us minimize the expression: E{llx- xI\2}= tr(E{(x-x)(x- X?})= 8. (9) Here E { ... } means the expected value and tr is the trace of the matrix. From the figure if follows that: (10) By substituting this into (9) we have: Fig. I; The condition of the minimum is that the derivative of (11) with respect to A equals zero: (12) Here we utilized the identities of the difterentiation according to a matrix and the exchangeability between the differentiation and the expected value formation. As E{ . .. } needs only be formed for the variable values, we obtained for the op-timal A: (13) Here the relationships (8) and the identity (AB)-I=B-IA -I were utilized. The relationship may also be written by the covariance matrices [1, 2]. By definition: Kx=E{(x-E{x))(x-E{x}}T} = =E{xxT}-E{x }E{x }Ta (14) 26 L. KONYA The quantities appearing in (13) are: Assuming E{ XZT} =E{x(x+ w)T}=E{nT}+E{xwT} E{ZZT}=E{xxT } +E{wwT}+E{xwT} +E{wxT }. 1 Ni ~ E{x}=- 2: xi=O /'.;[ i=1 and x, ware uncorrelated then With the above taken into account, (13) may be written in the form: (15) (16) (17) (18) (19) (20) Cmin is obtained by substituting Aopt into (9) and by performing the appointed opera-tions [1]: In the transformed domain the covariance rnatrix may also be determined. Since y=Fz Ky=E{yyT} - E{y }E{y V = FE{zZT}FT - FE{z }E{z }TFT I.e. Here Kz is the transformed co variance matrix. As K is a real synmletrical matrix, tr(K)= tr(FKFIt)= treK). By substituting (22) and (23) into (21): Cmin= tr(Kx - KAK-" + K,vl-1Kx)' (21) (22) (23) (24) (25) On the basis of (21), (24), (25) the value of Cmin is seen to be independent of the orthogonal transformation F. Accordingly it is advised to select F in a way that the transformation requires little computational work. From (10) it follows to be useful to select F in a way, that Aopt contains as few non-zero elements as possible, besides the data of the given random characteristics. DISCRETE ORTHOGONAL FUNCTIONS 27 Karhunen-Loeve transformation First let us examine the transformation F, which results in an optimal diagonal filter Aod ' Let us utilize the following theorem: If }'i and t; (i= 1, 2, ... , N) are the eigenvalues and the eigenvectors, respectively, of a real symmetrical matrix Q, then transform where (26) results in a diagonal matrix [4]. From (26) it is evident, that if F is chosen to be the eigenvector matrix of matrix Ar , then (27) is the desired optimum diagonal matrL'i:. The above orthogonal transformation whose basis vectors are eigenvectors of the specified Ar co variance-matrix is called the Karhunen-Loeve Transformation (KLT). Supoptimal diagonal filters Filters, Aopt=FArFT will be discussed which may be computed by the fast transformations. After transformation, the main diagonal elements of Aopt will be kept: From (20), using (23) - - -A[Kx+Kw]=Kx we get, keeping the diagonal elements of A: i=1,2, ., .,N where Kx(i, i)= FK"FT denotes the i-th diagonal element of Kx. Using (9), (21), (23) the mean square error is [1]: - -ed= tr(K,,)- tr(AdK,,) (28) (29) (30) (31) 28 L.KONYA but as Ad is diagonal (32) here we utilized (30). In the case of KL T : (33) where !Xi and Pi (i= 1,2, ... , N) are the eigenvalues ofKx and Kx+ Kw respectively. As a conclusion: the ideal transformation method is KLT, which results in an optimal diagonal filter, and the error (33) equals the mininmm value (25). The disadvantage of this method is that the eigenvectors of AI" must be determined, rather high in computation demand with increasing dimension, and the performance of the operation (10) requires a full matrix multiplication. In the case of suboptin1al diagonal filters, if fast transformation methods are used, the computational work decreases consider-ably, and the en-or will be as described by (32). The transformations can be compared on the basis of (32). The transformation is good if the value of (32) is low. Data Compression The basic problem is the following: vector x of N elements is given. By y = Fx transformation a new vector is obtained. Some of its elements are omitted before the inverse transformation, or replaced by selected elements. The objective is to select a subset of kf components of y, where lvf is substantially less than N. The remaining N - M components can then be discarded without introducing an objectionable error, when the signal is reconstmcted using the retained Jl1 components ofy. With (6a): N x=FTy= .2; Yli i=1 XT=[Xl' ... , XiV] (34) (35) (36) We retain the elements {Yi' i=l, 2, ... , M} (M<N), and substitute the N-M components ofy by preselected constants bi to obtain A1 lV x(M)= .2; Yifi+ .2; bifi (37) i=1 i=M+l where X'(M) denotes the estimate of x. The error vector: DISCRETE ORTHOGONAL FUNCTIONS N 8=X-X(M)= ~ (Yi-bj)fi. i=M+l Using (38) the mean square error is 8(M)=E{STs}=E t=~+l j=%+l (Yi- bi)(Yj-bj)f7fj} 29 (38) (39) 8(M) is seen to depend on the transformation F and on {bi}' The process of choosing the optima b; and f; is carried out in two steps. The optimum b; are obtained from (39) as follows: i=1v.[+1, .. . ,N. (40) But as (41) (40) may be written as: i=M+1, ... ,N. (42) Substituting (42) into (39) and using (41): N N 8(M)= ~ f7E{(x-E{x})(x-E{x})T}fj= ~ f?Kxfi . (43) i=M+l i=M+l Relationship (14) was utilized here. To obtain the optimum f; not only 8(M) has to be mininnzed with respect to fi' but also the constraint f7fi= 1 has to be satisfied. This may be performed with the help of the Lagrange multiplicator, and the following final result is obtained: (44) where Y; denotes the Lagrange multiplicator. By definition (44) implies that f; is the eigenvector of Kx and y; is the i-th corresponding eigenvalue. Here also the KLT transformation proves ideal. The error: N 8min(M)= ~ Yi' i=M+l (45) The matrix Ky=FKxFT will be diagonal if F is the matrix of KLT. From (45) it follows that, omitting a component Yk the mean square error increases by Yk' the corresponding eigenvalue. Thus the set of Y; with the greatest M eigenvalues should be selected, and the remaining Yi discarded since they can be substituted by the constants b;. From (42) it follows that b;= 0 if E{x}= O. For all other transformations, however, Ky has non-zero off diagonal terms. 30 L.KONYA Thus a logical criterion for selecting transformed components is to retain the set of M components with the greatest variances-the remaining N - Ai components can be discarded. This selection is called the variance criterion. As it has been men-tioned in the case of the Wiener filtering, the efficiency of the various transformations may be determined by error examination, here by the variance criterion. The presence of few elements of relatively great amplitudes in the main diagonal ofKy is favourable. For example, we have vectors of 128 elements. Storing these as words in a computer memory, every vector would occupy 128 places. With the help of this procedure, selecting the first 43 elements with the greatest variances, it is enough to store these, representing a 3-fold data compression without the change of the char-acter ofx at the resetting, i.e. "i is very similar to x". In pattern recognition problems, data compression corresponds to feature selection [1, 3]. Applications An important field of application of the method may be in computer aided medical diagnostics, where it may help classification and storage of various physiolog-ical curves (e.g. electrocardiograms). Another possibility is pattern recognition [1]. It may be used also in processing seizmogranls. It must be added, that the processing supposes availability of certain hardware, crucial for the utilization of the methods in practice. Summary The paper deals with the discrete orthogonal transformations, as a modern and useful tool of computer aided signal processing. In signal processing, a frequent problem is to separate the signal from the noise (filtering). Another problem may be the storage of the signal in the computer memory. The size of the memory may be reduced, if only the most important characteristics of the signal are stored. These tasks may be performed effectively by the application of orthogonal transformation. The purpose of this paper is a short review of this field and calling attention to the theme. References 1. ARMED, N.-RAo, K. R.: Orthogonal Transforms for Digital Signal Processing. Springer-Verlag, Berlin, 1975. 2. PRATI, W. K.: Generalized Wiener Filtering Computation Techniques. IEEE Trans. on Comp. vol C21 No. 7. July 1972. 3. NILSSON, N. J.: Learning Machines, McGraw-M.U New York 1965. 4. KORN, A. G.-KORN, T. M.: Mathematical Handbook for Scientists and Engineers. Miiszaki Konyvkiad6, Budapest 1975. (in Hung) 5. SZOKEFALvy-NAGY, B.: Real functions and Series of Functions. Tankonyvkiad6, Budapest, 1965. 6. Proc. 1972. Symp. Applications ofWalsh Functions. Naval Research Laboratory, Washington. Laszlo Konya, H-1521 Budapest 

DISCRETE ORTHOGONAL FUNCTIONS APPLIED TO FILTERING AND DATA COMPRESSION

https://pp.bme.hu/ee/article/download/4873/3978

DISCRETE ORTHOGONAL FUNCTIONS APPLIED TO FILTERING AND DATA COMPRESSION

Abstract

Similar works

Full text

Available Versions

Periodica Polytechnica (Budapest University of Technology and Economics)