Behavioral study of a system is an important task. It is mostly used in real world environments and became an emergent research area. Various approaches have been proposed since last two decades. In this paper, we have proposed a Variable Step-Size Sign-Sign Wilcoxon Approach, that is robust against outliers in the desired data and also convergence speed is faster than Wilcoxon norm based approach. In initial stage, Sign-Sign Wilcoxon norm based approach has been verified. Next to it, the proposed approach is verified and compared for the application in Linear and Non-linear system identification problems in presence of outliers.DOI:http://dx.doi.org/10.11591/ijece.v2i4.83

Dash, Sidhartha

Mohanty, Mihir Narayan

English

Institute of Advanced Engineering and Science

International Journal of Electrical and Computer Engineering (IJECE) Vol.2, No.4, August 2012, pp. 481~486 ISSN: 2088-8708  481   Journal homepage: http://iaesjournal.com/online/index.php/IJECE Variable Sign-Sign Wilcoxon Algorithm: A Novel Approach for System Identification   Sidhartha Dash1, Mihir Narayan Mohanty2 Department of Electronics and Communication Engineering, Institute of Technical Education and Research, Bhubaneswar, Odisha    Article Info  ABSTRACT  Article history: Received Jun 25, 2012 Revised Jul 23, 2012 Accepted Jul 28, 2012   Behavioral study of a system is an important task. It is mostly used in real world environments and became an emergent research area. Various approaches have been proposed since last two decades. We have verified the application of Wilcoxon norm in linear system identification in initial stage that follows for non-linear system. In the next stage, Sign-Sign Wilcoxon norm based approach has been verified for the same. Finally we have modified to a Variable Step-Size Sign-Sign Wilcoxon technique for both linear and non-linear system identification and compared with above two techniques. It has been observed that the proposed technique is robust against outliers in the desired data and simultaneously the convergence speed is faster than Wilcoxon norm based approach. Keyword: Adaptive Step-size Algorithm, Sign Wilcoxon Technique, System Identification, Variable step-size sign-sign Wilcoxon Technique. Wilcoxon norm,  Copyright © 2012 Institute of Advanced Engineering and Science.  All rights reserved. Corresponding Author: Sidhartha Dash Department of Electronics and Communication Engineering, Institute of Technical Education and Research,  Siksha ‘O’ Anusandhan University, Jagamara, Bhubaneswar, Odisha. Email: sidharthadashiter@gmail.com   1. INTRODUCTION The boundaries of a real environment cannot be defined correctly. The only thing the model-designer can do is to define a modeldescribing the behaviour of the environment to be modelled with noinconsistencies at the time the model is constructed. Therefore each model basedon real environments needs to be redesigned continuously as the time passes toprovide their consistency. It is only feasible using adaptive knowledge-intensivemodelling tools.Signal processing has exploited a major role in this context.Technical advancements such as noise filtering, system identification, channel estimation, detection and voice prediction task explore in such area. Adaptive techniques are the solution to promote accuracy and convergence to the solution.System identification is one of the important aspects for designing a system using adaptive algorithm. The identification task covers almost all the area of engineering application such as problem of building models of systems [1]. Identification aims at finding a mathematical model from the measurement record of inputs and outputs of a system. In this problem, the task is to determine a suitable estimate of finite dimensional parameters, which completely characterize the non-linear unknown plant. The selection of the estimate is based on comparison between the actual output sample and a predicted value on the basis of input data up to that instant [2]. The standard block diagram for system identification is shown in Figure 1. The input is applied to the plant as well as to the adaptive model that is parallel to the unknown system. The impulse response of the linear segment of the plant is represented by w(n), which is followed by non-linearity (NL) associated with it. White Gaussian noise g(n)is added with nonlinear output accounts for measurement noise. The desired output d(n)is compared with the estimated output y(n) of the identifier to generate the error e(n), and is used by the new adaptive algorithm for updating the weights of the model h(n).                  ISSN: 2088-8708  IJECE Vol. 2, No. 4, August 2012 : 481–486 482   Figure 1. Block Diagram for Non-Linear System Identification   The objective is to minimize the error e(n) by using the algorithm that occurs at a particular set of weights. These weightsare the optimized value for a solution. It can be found by using,  ( ) ( )opth n h n for= n → ∞       (1)  where,hopt(n) is an optimum set of filter coefficients for the model at time n. In case of optimized weights, the adaptive filter model provides a best performance.  In most of the identification problems, the training data suffers with mislabeling, biases, imbalance, impulse noise, outliers, etc. Depending on their location, outliers may have moderate to severe effects on the regression model [3-4]. The conventional LMS adaptive algorithm is not efficient against outliers in the training signal. In terms of convergence speed and robustness towards the outliers and impulse noise, the LMS algorithm has no such desired result [5-6].Wilcoxon Learning Model is one of the effective methods of robust identification in presence of outliers. The weights of the models are iteratively updated, which progressively reduces the norm. Wilcoxon norm is best suitable for regression analysis [6]. A simple and efficient normalized Sign-Sign LMS algorithm is sufficient for applications requiring large signal to noise ratios with less computational complexity and faster convergence speed compared to conventional LMS [7]. Moreover fixed step-size general recursive adaptive algorithms are weakly converging and locally stable [8]. These types of adaptive algorithms used to track slowly moving parameters.  This motivates to propose avariable sign-signWilcoxon learning algorithm used for linear and non-linear system identification. The paper is organized as follows: Section 2 describes the research method that includes the study of Wilcoxon learning algorithm and establishes the proposed step-size sign sign Wilcoxonalgorithm, used to determine the optimum filter coefficients in system identification problem. Section 3 discusses the extensive simulation results in linear as well as non-linear system identification in presence of outliers and section 4 concludes the work.   2. RESEARCH METHOD 2.1. Learning Algorithm It is well known that for any adaptive system learning is required initially to train the system. The perfect learning will result of adaptability. In case of normalized LMS, the norm can be used for various applications. Wilcoxon norm is one of them for excellent performance.Wilcoxon Algorithm isaneffective methodfor robust identification [3]. The cost function of the proposed model is formulated as Wilcoxon norm. The Wilcoxon Norm of a vector is analyzed in terms of an increasing score function and is defined as  Ф( ) : [0,1]u R→         (2)   And the score function is standardized such that 120Ф ( ) 1u du =∫         (3) And10Ф( ) 0u du =∫        (4) Let the error vector of pth particle at kth generation due to application of  N input samples to the model be represented as [e1,p(k),e2,p(k),…….eN,p(k)]T. The error vector is obtained by subtracting estimated IJECE ISSN: 2088-8708    Variable Sign-Sign Wilcoxon Algorithm: A Novel Approach For System Identification (Dash S) 483  output from the desired output. The errors are then sorted in an ascending order from which the rank of each error term is obtained [6]. The score associated with each error term can be evaluated as ( ) Ф ( ) 12 ( )12 0.51a i u uiN= = = − +        (5) where 0.51iuN = − +        (6)   Here Nis a fixed positive number that defines the length of the error vector and i denotes the rank associated with each error term (1 ≤ i ≤ N). At kth generation of each pth particle the Wilcoxon norm is calculated as ,1( ) ( ) ( )Np i piC k a i e k==∑       (7) In gradient based technique, the cost functionCp(k) is formulated as the Wilcoxon Norm of the error vector.The cost function in (7) is robust to impulse noise as well as outliers in the system [9].  The weight update equation for the Learning Algorithm is  ( 1) ( ) 2 ( ) 12( )h n h n x n uµ+ = +       (8)  Where µ  is defined as the fixed step size and x(n) is the random input samples.  2.2 Variable Step Size Sign Sign Wilcoxon Algorithm The objective is to design a model and estimate its parameters effectively and efficiently in presence of outliers [9]. Both Wilcoxon norm and sign Wilcoxon norm have exhibited the robustness for the impulse noise as well as outliers [3-4]. It is observed that sign-sign techniques are faster in terms of convergence compared to conventional methods. In Sign-Sign Wilcoxon approach both error vector and score matrix is represented in terms of its signum function.  A Signum function can be represented as ( ) ( )y f t sign t= =       (9) where, an array y is same size as that of t and each element of y are  1, if the corresponding element of xt>0.  0, if the corresponding element of   xt =0. -1, if the corresponding element of   xt<0. For non-zero complex input the function is defined as  ( ) / ( )y sign t t abs t= =         (10)  The modified expression for Sign-Sign Wilcoxon Algorithm is  ( 1) ( ) 2 ( ) 12 ( ) ( ( ))h n h n x n sign u sign e nµ+ = +    (11)  It describes the weight update equation using a fixed step-size parameter µ . The fixed step-size adaptive algorithms are effective only to track a slowly moving parameterization and then work efficiently for linear systems [8]. However most of the real time systems are non-linear, so the fixed step-size Sign Sign Wilcoxon Approach doesn’t produce better result compared to variable step-size.  To construct even more robust systemthe adaptive algorithm is modified with variable-step-size. As step-size parameter plays a vital role, the variation of it can be applied for different applications. Mostly the non-linear system it is advantageous. The instantaneous step-size is defined as Re( . )Tins prevs sµ µ µ= + ∆       (12) where, µ ins is the instantaneous step-size value which is used in the proposed adaptive algorithm µ is termed as the initial value of the fixed step size and  ∆µ is the increment by which the step size changes from iteration to iteration. ( ). ( )s x n e n=  and, prevs  is the value of s in the previous iteration.                 ISSN: 2088-8708  IJECE Vol. 2, No. 4, August 2012 : 481–486 484  And the final Step-size is defined as ( ) insnµ µ=, if   min maxinsµ µ µ< <       minµ=,        if    mininsµ µ<       maxµ=,        if    maxinsµ µ>  where,minµ  and maxµ are defined as the MinStep and MaxStep for the proposed Algorithm. The step size can vary within the values of minµ  and maxµ .  The final update equation for the Variable Step-size Sign-Sign Wilcoxon Algorithm is considered as ( 1) ( ) 2 ( ) ( ) 12 ( ) ( ( ))h n h n n x n sign u sign e nµ+ = +    (13) The proposed technique is verified for linear and non-linear system identification problem in presence of outliers. The simulation results are compared with Wilcoxon techniques for linear as well as for non-linear system.   3. RESULTS AND DISCUSSION It has been simulated with Step-Size parameter µ  as 0.02. 1000 random value between [-0.5 0.5] is considered forinput. The functiong(n) is additive white Gaussian noise having SNR 15db. The proposed weight update equation is used for linear and non-linear system identification. Analysis has been carried without outliers as well as in presence of outliers and the simulation results are compared with Wilcoxon Technique. The simulation result is shown between number of iterations and normalized MSD (Mean Square Deviation) in db. The original weights of the system is considered as w = [0.26 0.93 0.26].The workhas been performed using MATLAB-7 environment.  3.1. Linear system (NL=0):  3.1.1. Comparison of actual weight with the estimated weight    Figure 2. Comparison of the actual weight with the estimated weight without outliers     Figure 3.Comparison of the actual weight with the estimated weight in presence of outliers    Figure 2 and 3compares the actual weight with the estimated weight of the 3-tap adaptive filter in case of linear system identification. From the figures it is found that, the estimated weight of the filter is very close to the actual weight using the proposed variable step-size Wilcoxon technique compared to wilcoxon norm. For the noisy and non-linear systems the deviation parameter plays a major role for convergence of algorithm. It has the effect in cost function.  3.1.2. MSD analysis Figure 4-5 is MSD curves for linear system identification. From the figures, it is found that the proposed variable step-size Wilcoxon technique is robust against outliers and convergence speed is faster than the Wilcoxon norm as well as sign Wilcoxon norm.  IJECE ISSN: 2088-8708    Variable Sign-Sign Wilcoxon Algorithm: A Novel Approach For System Identification (Dash S) 485    Figure 4. MSD Analysis without outliers in linear system identification    Figure 5. MSD Analysis in presence of outliers in linear system identification   3.2. Non-linear system : In most of the Non-Linear system Identification problem, the process model is usually described as differential equations (continuous-time model) or difference equations (discrete-time model) [10]. In this approach, it has beenverified the proposed Algorithm for a standard non-linear equation based system.  Non-linear system Equation ( ) tanh( ( ))Td n xw g n rand= + +      (14)   Figure 6. MSD Analysis without outliers in non-linear system identification   Figure 7. MSD Analysis in presence of outliers in non-linear system identification  Figure 6-7 is MSD curves for Non-linear system identification having non-linearity given in (14). In this case the performance is degrading with comparison to linear system as the MSD curve converges at -50db in non-linear system compared to -70db in linear case. But in comparison with the Wilcoxon and Sign-Wilcoxon techniques, the proposed technique is showing good performance with faster convergence as it converges with less number of iterations. As well as the proposed technique is robust against outliers.   4. CONCLUSION In this paper, the Wilcoxon norm based approach for system identification is considered basically. The new approach has been derived and applied for both linear and non-linear system identification that is applicable for the real world problem. The simulated result shows its performance against outliers as well as for the convergence. Tuning of parameters may be required for further accuracy and convergence. Also it may be tested for various applications including the industrial application. System identification is most suitable for automatic control and study of the dynamic systems. Also, one of the example is, Using system                 ISSN: 2088-8708  IJECE Vol. 2, No. 4, August 2012 : 481–486 486  identification techniques to develop a mathematical model which describes the dynamics in the irrigation system is a helpful tool for control system design.   REFERENCES [1] Simon Haykin,“Adaptive Filter Theory ”, Chapter 20,p 855-874, PHI Publication. [2] Kumpati  S. Narendra, and Kannan Parthasarathy, “Identification and Control of Dynamical Systems using neural networks”, IEEE Transactions on Neural Networks,  vol.1,No.1.March 1990.  [3] Sidhartha Dash and Mihir Narayan Mohanty, “Analysis Of Outliers In System Identification Using Wlms Algorithm”, IEEE International Conference on Computing, Electronics and Electrical Technologies [ICCEET],Kanyakumari, 21-22 March,2012. [4] Dingsheng Luo, Xinhao Wang, Xihong Wu,and  Huisheng Chi, “Outliers Learning and Its Applications”, IEEE conference,  2005.  [5] Simon Haykin,“Neural Networks”, A Comprehensive Foundation 2nd Edition, Pearson Education.  [6] Sidhartha Dash and Mihir Narayan Mohanty, “A Comparative Analysis for Wlms Algorithm in System Identification”,IEEE International Conference on Electronics Computer Technology (ICECT 2012), Kanyakumari, 06-07,April,2012. [7] R.A. Shaik and D.V.R.K  Reddy, “Noise cancellation in ECG signals using normalized Sign-Sign LMS algorithm”, IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), 14-17 Dec, 2009.  [8] James A. Bucklew, Thomas G. Kurtz and William A. Sethares, “Weak Convergence and Local Stability Properties of Fixed Step Size Recursive Algorithms”, IEEE Transactions on Information Theory, Vol. 30, No. 3, MAY 1993. [9] Hseih,Lin, and Jeng, “Preliminary Study  on Wicoxon Learning Machines”, IEEE Transactions on Neural Networks,vol.19, No.2 Feb 2008.  [10] Yong Liu and J. Jim Zhu,“Continuous-time Nonlinear System Identification Using Neural Network”,American Control Conference Westin Seattle Hotel, Seattle, Washington, USA,June 11-13, 2008.   BIOGRAPHIES OF AUTHORS     Sidhartha Dash is presently working as an Assistant Professor in the Department of Electronics and Communication Engineering, Institute of Technical Education and Research, Bhubaneswar, Odisha. He obtained his M-Tech degree from KIIT, BBSR in 2010. He has 5 papers in International/National Journals and conferences. His area of research interests includes – Digital signal Processing  and Image Processing. E-mail id: sidharthadashiter@gmail.com       Mihir N. Mohanty is presently working as an Associate Professor in the Department of Electronics and Communication Engineering, Institute of Technical Education and Research, Bhubaneswar, Odisha. He has over 30 papers in International/National Journals and conferences. His area of research interests includes – Digital signal/image processing, Biomedical signal processing and Bioinformatics. E-mail id: mihir.n.mohanty@gmail.com  

Variable Sign-Sign Wilcoxon Algorithm: A Novel Approach for System Identification

IAES journal

http://ijece.iaescore.com/index.php/IJECE/article/download/5354/4369

Variable Sign-Sign Wilcoxon Algorithm: A Novel Approach for System Identification

Abstract

Similar works

Full text

Available Versions

Institute of Advanced Engineering and Science

IAES journal