This study investigates the ability of recursive least squares (RLS) and least mean square (LMS) adaptive filtering algorithms to predict and quickly track unknown systems. Tracking unknown system behavior is important if there are other parallel systems that must follow exactly the same behavior at the same time. The adaptive algorithm can correct the filter coefficients according to changes in unknown system parameters to minimize errors between the filter output and the system output for the same input signal. The RLS and LMS algorithms were designed and then examined separately, giving them a similar input signal that was given to the unknown system. The difference between the system output signal and the adaptive filter output signal showed the performance of each filter when identifying an unknown system. The two adaptive filters were able to track the behavior of the system, but each showed certain advantages over the other. The RLS algorithm had the advantage of faster convergence and fewer steady-state errors than the LMS algorithm, but the LMS algorithm had the advantage of less computational complexity

Tajdari, Teimour

English

University of Niš: Facta Universitatis (E-Journals) / Универзитет у Нишу

FACTA UNIVERSITATIS Series: Electronics and Energetics Vol. 34, No 1, March 2021, pp. 133-140 https://doi.org/10.2298/FUEE2101133T © 2021 by University of Niš, Serbia | Creative Commons License: CC BY-NC-ND Original scientific paper ADAPTIVE METHOD TO PREDICT AND TRACK UNKNOWN SYSTEM BEHAVIORS USING RLS AND LMS ALGORITHMS Teimour Tajdari Electrical Engineering Department, Engineering Faculty, Velayat University,  Iranshahr, Iran Abstract. This study investigates the ability of recursive least squares (RLS) and least mean square (LMS) adaptive filtering algorithms to predict and quickly track unknown systems. Tracking unknown system behavior is important if there are other parallel systems that must follow exactly the same behavior at the same time. The adaptive algorithm can correct the filter coefficients according to changes in unknown system parameters to minimize errors between the filter output and the system output for the same input signal. The RLS and LMS algorithms were designed and then examined separately, giving them a similar input signal that was given to the unknown system. The difference between the system output signal and the adaptive filter output signal showed the performance of each filter when identifying an unknown system. The two adaptive filters were able to track the behavior of the system, but each showed certain advantages over the other. The RLS algorithm had the advantage of faster convergence and fewer steady-state errors than the LMS algorithm, but the LMS algorithm had the advantage of less computational complexity. Key words: RLS algorithm, Steepest decent algorithm, LMS algorithm, System identification 1. INTRODUCTION The family of adaptive filters has long been used for system identification [1, 2]. Identifying and predicting unknown systems is important when their behavior affects other cooperative systems [3]. This technique has application in tension control in mechanical and civil engineering, as well as in robots and autonomous vehicles. Least Mean Square (LMS) and Recursive Least Squares (RLS) are useful tools for identifying and tracking the behavior of unknown systems. LMS and RLS are adaptive filtering algorithms developed based on Wiener's filter theory [4, 5]. The adaptive filter can adjust and correct filter coefficients according to changes in unknown system parameters[6].  Received August 22, 2020; received in revised form December 15, 2020 Corresponding author: Teimour Tajdari Department of Electrical Engineering, Engineering Faculty, Velayat University, Iranshahr, Iran E-mail: tajdari.t@velayat.ac.ir 134 T. TAJDARI The algorithm regularly adjusts the filter coefficients for the incoming samples of the input signal, reducing the difference error between the filter output and the unknown system output at each iteration to the best predicted coefficient that minimizes the error. Bernard Widrow of Stanford University invented the LMS algorithm in 1959. It follows the steepest descent algorithm in which the filter is adjusted according to the current time error [7]. Mukhopadhyay S. et al. developed the LMS algorithm to solve the problem of missing data. They used an unknown input data product with i.i.d. Bernoulli's sequence of random variables for modeling input data [8]. To enhance estimation for sparse channel estimation usage, sparse least‐mean mixed‐norm technique has been created [9, 10]. Eleyan et al. studied the convergence behavior of the MN-LMS algorithm, which performed significantly better than other algorithms at different sparsity and SNR [11]. Dogariu LM. et al provided a simple adaptive algorithm for the identification of nonlinear systems based on the LMS algorithm, where the Taylor series expansion was used together with LMS to determine the nonlinearities of the system [1, 12]. The RLS is an adaptive algorithm that recursively computes coefficients to minimize the weighted linear least squares cost function associated with the input signal. Ding F. et al. presented a method based on the identification of the auxiliary model where the unknown coefficients in the data vector are replaced with their estimates, which are computed through the estimates of the previous parameters [5, 13, 14]. Mattson P. et al. developed a technique to learn nonlinear models with multiple outputs and inputs so that predictive errors are modeled for the system by applying a latent variable framework. Then, convex majorization principles were applied to perform a recursive identification method [15, 16]. Elisei C. et al. studied the RLS algorithm to identify systems with large parameter spaces. They defined the bilinearterm taking into account the impulse response of the model. They also proposed a variable-regularized estimation model that self-adjusts the coefficients by estimating the signal-to-noise ratio [17, 18]. In this study, LMS and RLS adaptive algorithms were examined to identify the same unknown system in order to compare their performance. The methodology, results and discussions, and conclusion of this study are provided below. 2. METHODOLOGY To identify the unknown system, as shown in Fig. 1, the same input signal x(n) is provided to the system and the adaptive filter. The adaptive algorithm adjusts the filter coefficients such that the difference error e(n) between the output of the filter y(n) and the output of the system s(n) approaches zero at each iteration of the algorithm. When the error reaches zero, y(n) becomes similar to the unknown system output. In this study, a system that functions as a bandpass filter using an IIR filter was designed to represent an unknown system. In addition, an audio signal representing the input signal x(n) was provided. The adaptive filter was designed using LMS and RLS algorithms, then inspected one  Fig. 1 Adaptive system identification block diagram  Adaptive Method to Predict and Track Unknown System Behaviors Using RLS and LMS Algorithms 135 by one to identify unknown systems, as shown in Fig. 1, and then the results of both algorithms were collected and analyzed to compare the performance of each algorithm. 2.1. System Identification Using LMS Algorithm LMS is an adaptive filter algorithm used to self-adjust filter coefficients to generate the least squares error between the output and the desired signal. FIR filter y(n) with filter coefficient 0 1 -1[   ... ]TNW w w w=  is given by  ( ) ( )Ty n W X n=  (1) where ( ) [ ( ) ( -1) ... ( - 1)]TX n x n x n x n N= +  is the vector of N input signal samples. The least mean square error between system output s(n) and filter output y(n) can be given as follows [19].  2 22J wP w R= − +  (2) where J represents mean square error,  2 is the power of s(n), P represents the cross-correlation between s(n) and X(n), and R is the autocorrelation of X(n). The dJ / dW is calculated as follows.  2 ( ) ( )( )dJe n X ndw n= −  (3) Assuming Eq. 3 in the steepest descent algorithm, the best W(n) for the next iteration is:  ( 1) ( ) 2 ( ) ( )W n W n e n X n+ = +  (4) where  is the convergence factor and for a 16-bit ADC converter it would be as follows [20].  3012N =+ (5) The LMS algorithm requires initializing vector W(n) with arbitrary values, computing e(n) = s(n)  − y(n), and then computing W(n+1) for the next iteration. As the iteration continues, the ( )e n  approaches zero. 2.2. System Identification Using RLS Algorithm In Eq. 2, the best approximation to the filter coefficients can be obtained by solving dJ / dW = 0 for W, and the result is expressed as W * = R−1P. However, in practical applications, it is impossible to calculate R−1 for many coefficients. The RLS algorithm uses the matrix inversion lemma to handle R−1 computational aspects [20, 21]. In this method, R  and P  are calculated in recursive for, that is,  ( ) ( 1) ( ) ( )TR n R n X n X n= − +  (6)  ( ) ( 1) ( ) ( )P n P n s n X n= − +  (7) where the   is called the weighting factor that gives less weight to older error samples exponentially and obtained empirically as 0 1  . Solving / 0dJ dW =  over W gives:  ( ) ( 1) ( ) ( )W n W n k n n= − +  (8) 136 T. TAJDARI In which, ( )n  and ( )k n  are calculated as follows.  ( ) ( ) ( ) ( 1)Tn s n X n W n = − −  (9)  11( 1) ( )( )1 ( ) ( 1) ( )TQ n X nk nX n Q n X n−−−=+ − (10) where,   1 1( ) ( 1) ( ) ( ) ( 1)TQ n Q n k n X n Q n − −= − − −  (11) To perform the RLS algorithm, it is initially necessary to determine arbitrary initial values for the vector of coefficients W(n) when n = 0, as well as to calculate Q(n −1) =  I where I is the identity matrix and  is the inverse of the power of X(n). In each iteration, you need to calculate k(n) and (n) for W(n) as in Eq. 9 and Eq. 10, then Q(n) as in Eq. 11. After calculating e(n) = s(n) − y(n), the problem continues in the next iteration. As the iterations continue, e(n) approaches zero and finally y(n) follows s(n) exactly. 3. SIMULATIONS, RESULTS AND DISCUSSIONS A test unknown system was configured for the simulation of adaptive identification. An input signal was provided for the process, and then the two algorithms, LMS and RLS, were used to identify the unknown system, separately. 3.1. Presentation of the Test Unknown System The test unknown system was a 12-order IIR bandpass filter with a passband frequency of 500Hz to 1000Hz and ripple of 1dB, and stopband attenuation of 40dB. Fig. 2 shows the input signal x(n) and its frequency response, filter frequency spectrum, and the resulting signal filtering spectrum.  Fig. 2 The input test signal ( )x n , and the test unknown system as a bandpass filter  Adaptive Method to Predict and Track Unknown System Behaviors Using RLS and LMS Algorithms 137 3.2. The LMS Algorithm Implementation Results To perform adaptive identification using the LMS algorithm, the filter tap was selected to be 51 and the filter coefficient vector W(n) was initialized to zero. If the convergence factor  is chosen to be large, the stability of the filter is weakened. On the other hand, if a very small  is selected, the convergence time increases considerably. Here, the  was empirically adjusted to 0.036. The Fig. 3 shows the LMS results.  Fig. 3 The LMS algorithm system identification results In Fig. 3, the upper signal graph shows the first 700 samples for the unknown system output and the LMS adaptive filter output. The results show it takes the LMS algorithm about 400 samples to simulate the behavior of the unknown system. The middle signal graph shows 700 output samples since 40k samples have already passed, and the adaptive identifier indicates that it operates with a low error in a stable state. The signal graph at the bottom of the figure shows the error between the unknown system output and the LMS adaptive filter output at about 80k samples, and shows the error within 10 seconds since the sampling rate is 8000Hz. As shown, identification has a large magnitude of error at first and decreases as identification continues, but it always exists.  3.3. The RLS Algorithm Implementation Results The RLS adaptive identification algorithm was implemented using the same configuration and the same input signal used in the application of the LMS algorithm. The filter tap was selected 51 and the filter coefficient vector W(n) was initialized to value of 0. The  weighting factor should not be too small since the lower weight leads to instability. It was decided experimentally to its best performance, which was 0.94. Figure 4 shows the results of the implementation of the RLS algorithm. 138 T. TAJDARI  Fig. 4 The RLS algorithm system identification results In Fig. 4, the top signal graph indicates the first 700 signal samples from the unknown system output and the RLS filter output. As shown, in 51 first sample, the filter output is zero, but then it begins to follow the behavior of the unknown system precisely. The signal graph in the center of the figure shows another 700 signal samples from the unknown system output and the RLS filter output, but as the 40k sample has already passed. It shows the RLS filter output perfectly overlaps the system output sample, so the system is accurately identified. The signal graph at the bottom of the figure shows the error within 10 seconds of system identification, which shows a very small error. 3.4. Discussion of Results As the results show, the RLS algorithm showed much better system identification ability than the LMS algorithm. The LMS algorithm took longer to converge than the RLS algorithm. Also, the steady state error of the LMS algorithm was much larger than that of the RLS algorithm. This may be due to differences in algorithmic characteristics, where the LMS algorithm relies on the steepest descent method to converge filter coefficients to achieve optimized filter weights. The RLS algorithm, on the other hand, finds filter coefficients in a recursive manner by minimizing the weighted linear least squares loss function associated with the input signal. Therefore, this algorithm involves data from the starting point to the current point. It makes the RLS algorithm generate better results compared to the LMS algorithm but at the cost of greater computational complexity.   Adaptive Method to Predict and Track Unknown System Behaviors Using RLS and LMS Algorithms 139 4. CONCLUSION This study compared the performance of two major adaptive filters, the LMS algorithm and the RLS algorithm, on the subject of identifying unknown systems. System identification is important for controlling and modifying the behavior of the system. Adaptive filters have the ability to self-adjust coefficients to mimic the behavior of different systems for a similar input signal. Adaptive LMS and RLS filters were designed to detect an unknown system separately. Both algorithms were tested using an input signal intended for both the adaptive filter and the unknown system. The results showed the great performance of both algorithms for system identification. However, the RLS filter provided the results with a smaller identification error in contrast to the LMS algorithm but at the cost of increased computational complexity. REFERENCES    [1] L. M. Dogariu, S. Ciochină, C. Paleologu, J. Benesty, and P. Piantanida, "An adaptive solution for nonlinear system identification", In Proceedings of the International Symposium on Signals, Circuits and Systems (ISSCS), Iasi, Romania, 2017, pp. 1-4.   [2] P. Kshirsagar, D. Jiang, and Z. Zhang, "Implementation and Evaluation of Online System Identification of Electromechanical Systems Using Adaptive Filters", IEEE Trans. Ind. Appl., vol. 52, no. 3, pp. 2306-2314, 2016.   [3] A. Abid, M. T. Khan, H. Lang, and C. W. d. Silva, "Adaptive System Identification and Severity Index-Based Fault Diagnosis in Motors", IEEE/ASME Trans. Mechatron., vol. 24, no. 4, pp. 1628-1639, 2019.   [4] S. Ciochină, C. Paleologu, J. Benesty, S. L. Grant, and A. Anghel, "A family of optimized LMS-based algorithms for system identification", In Proceedings of the 24th European Signal Processing Conference (EUSIPCO), Budapest, Hungary, 2016, pp. 1803-1807.   [5] F. Ding, X. Liu, and M. Liu, "The recursive least squares identification algorithm for a class of Wiener nonlinear systems", J. Franklin Inst., vol. 353, no. 7, pp. 1518-1526, 2016.   [6] M. T. M. Silva, R. Candido, J. Arenas-Garcia, and L. A. Azpicueta-Ruiz, "Improving Multikernel Adaptive Filtering with Selective Bias", In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, 2018, pp. 4529-4533.   [7] S. R. Prasad and S. A. Patil, "Implementation of LMS algorithm for system identification", In Proceedings of the International Conference on Signal and Information Processing (IConSIP), Vishnupuri, 2016, pp. 1-5.   [8] S. Mukhopadhyay and A. Mukherjee, "ImdLMS: An Imputation Based LMS Algorithm for Linear System Identification With Missing Input Data", IEEE Trans. Signal Process., vol. 68, pp. 2370-2385, 2020.   [9] Y. Li, Y. Wang, and T. Jiang, "Sparse least mean mixed-norm adaptive filtering algorithms for sparse channel estimation applications", Int. J. Commun. Syst., vol. 30, no. 8, p. e3181, 2017. [10] W. Ma, X. Qiu, J. Duan, Y. Li, and B. Chen, "Kernel recursive generalized mixed norm algorithm", J. Franklin Ins., vol. 355, no. 4, pp. 1596-1613, 2018. [11] G. Eleyan and M. S. Salman, "Convergence analysis of the mixed-norm LMS and two versions for sparse system identification", Signal, Image and Video Processing, vol. 14, no. 5, pp. 965-970, 2020. [12] L.-M. Dogariu, S. Ciochină, J. Benesty, and C. Paleologu, "System Identification Based on Tensor Decompositions: A Trilinear Approach", Symmetry, vol. 11, no. 4, p. 556, 2019. [13] F. Ding and T. Chen, "Identification of Hammerstein nonlinear ARMAX systems", Automatica, vol. 41, no. 9, pp. 1479-1489, 2005. [14] F. Ding, X. Wang, Q. Chen, and Y. Xiao, "Recursive Least Squares Parameter Estimation for a Class of Output Nonlinear Systems Based on the Model Decomposition", Circ. Syst. Signal Pr., vol. 35, no. 9, pp. 3323-3338, 2016. [15] P. Mattsson, D. Zachariah, and P. Stoica, "Recursive nonlinear-system identification using latent variables", Automatica, vol. 93, pp. 343-351, 2018. [16] P. Mattsson, D. Zachariah, and P. Stoica, "Identification of cascade water tanks using a PWARX model", Mech. Syst. Signal Process., vol. 106, pp. 40-48, 2018. [17] C. Elisei-Iliescu, C. Paleologu, C. Stanciu, C. Anghel, S. Ciochină, and J. Benesty, "Regularized Recursive Least-Squares Algorithms for the Identification of Bilinear Forms", In Proceedings of the International Symposium on Electronics and Telecommunications (ISETC), Timisoara, 2018, pp. 1-4. 140 T. TAJDARI [18] C. Elisei-Iliescu, C. Stanciu, C. Paleologu, J. Benesty, C. Anghel, and S. Ciochină, "Efficient recursive least-squares algorithms for the identification of bilinear forms", Digit. Signal Process., vol. 83, pp. 280-296, 2018. [19] A. D. Poularikas, "The Least Mean Square Algorithm", in Adaptive Filtering Fundementals of Least Mean Squares with MatlabUnited State of America: CRC Press, 2015, pp. 203-232. [20] L. Tan and J. Jiang, "Adaptive filters and aplications", in Digital Signal Processing Fundamentals and Applications 3rd ed. USA: Academic Press, 2019, pp. 421-474. [21] R. J. Schilling and S. L. Harris, "Adaptive Signal Processing", in Digital Signal Processing Using MATLABUnited States of America: Cengage Learning, 2017, pp. 667-760.  

ADAPTIVE METHOD TO PREDICT AND TRACK UNKNOWN SYSTEM BEHAVIORS USING RLS AND LMS ALGORITHMS

http://casopisi.junis.ni.ac.rs/index.php/FUElectEnerg/article/download/6864/4074

ADAPTIVE METHOD TO PREDICT AND TRACK UNKNOWN SYSTEM BEHAVIORS USING RLS AND LMS ALGORITHMS

Abstract

Similar works

Full text

Available Versions

University of Niš: Facta Universitatis (E-Journals) / Универзитет у Нишу