Stochastic Behavior Analysis of the Gaussian Kernel Least-Mean-Square Algorithm

The kernel least-mean-square (KLMS) algorithm is a popular algorithm in nonlinear adaptive filtering due to its simplicity and robustness. In kernel adaptive filters, the statistics of the input to the linear filter depends on the parameters of the kernel employed. Moreover, practical implementations require a finite nonlinearity model order. A Gaussian KLMS has two design parameters, the step size and the Gaussian kernel bandwidth. Thus, its design requires analytical models for the algorithm behavior as a function of these two parameters. This paper studies the steady-state behavior and the transient behavior of the Gaussian KLMS algorithm for Gaussian inputs and a finite order nonlinearity model. In particular, we derive recursive expressions for the mean-weight-error vector and the mean-square-error. The model predictions show excellent agreement with Monte Carlo simulations in transient and steady state. This allows the explicit analytical determination of stability limits, and gives opportunity to choose the algorithm parameters a priori in order to achieve prescribed convergence speed and quality of the estimate. Design examples are presented which validate the theoretical analysis and illustrates its application

PARTICLE SWARM OPTIMIZATION AND NEURAL NETWORK FOR FREQUENCY DOMAIN IDENTIFICATION OF SERVO SYSTEM WITH FRICTION FORCE

ABSTRACT: Generally, the mechanical devices come with undesirable nonlinearities. Due to these nonlinearities the frequency domain system identification process in servo system seems to be a tough task. In the paper, particle swarm optimization (PSO) algorithm based hybrid technique is proposed for the frequency domain identification of servo system. The proposed hybrid technique is the combination of artificial neural network (ANN) and PSO algorithm. Initially, the system parameters are generated as a data set at different mass level by the artificial network. From the dataset, the PSO algorithm is used to optimize the system parameters such as pole, constant, DC gain and friction force etc. Then, the optimized parameters are applied to the system and the friction of system is analyzed in terms of velocity. The proposed identification method is implemented in MATLAB working platform and the deviation performances are evaluated. The system parameters identified by proposed method (PSO-ANN) is compared with actual system, GA-ANN, and adaptive GA-ANN

On Recovering Missing Ground Penetrating Radar Traces by Statistical Interpolation Methods

Missing traces in ground penetrating radar (GPR) B-scans (radargrams) may appear because of limited scanning resolution, failures during the acquisition process or the lack of accessibility to some areas under test. Four statistical interpolation methods for recovering these missing traces are compared in this paper: Kriging, Wiener structures, Splines and the expectation assuming an independent component analyzers mixture model (E-ICAMM). Kriging is an adaptation to the spatial context of the linear least mean squared error estimator. Wiener structures improve the linear estimator by including a nonlinear scalar function. Splines are a commonly used method to interpolate GPR traces. This consists of piecewise-defined polynomial curves that are smooth at the connections (or knots) between pieces. E-ICAMM is a new method proposed in this paper. E-ICAMM consists of computing the optimum nonlinear estimator (the conditional mean) assuming a non-Gaussian mixture model for the joint probability density in the observation space. The proposed methods were tested on a set of simulated data and a set of real data, and four performance indicators were computed. Real data were obtained by GPR inspection of two replicas of historical walls. Results show the superiority of E-ICAMM in comparison with the other three methods in the application of reconstructing incomplete B-scans.This research was supported by Universitat Politecnica de Valencia (Vice-Rectorate for Research, Innovation and Transfer) under Grant SP20120646; Generalitat Valenciana under Grants PROMETEOII/2014/032, GV/2014/034 (Emergent Research Groups), and ISIC/2012/006; and the Spanish Administration and European Union FEDER Programme under Grant TEC2011-23403.Safont Armero, G.; Salazar Afanador, A.; Rodriguez, A.; Vergara Domínguez, L. (2014). On Recovering Missing Ground Penetrating Radar Traces by Statistical Interpolation Methods. Remote Sensing. 6(8):7546-7565. https://doi.org/10.3390/rs6087546S7546756568Le Bastard, C., Baltazart, V., Yide Wang, & Saillard, J. (2007). Thin-Pavement Thickness Estimation Using GPR With High-Resolution and Superresolution Methods. IEEE Transactions on Geoscience and Remote Sensing, 45(8), 2511-2519. doi:10.1109/tgrs.2007.900982Schafer, R. W., & Rabiner, L. R. (1973). A digital signal processing approach to interpolation. Proceedings of the IEEE, 61(6), 692-702. doi:10.1109/proc.1973.9150Salazar, A., Vergara, L., Serrano, A., & Igual, J. (2010). A general procedure for learning mixtures of independent component analyzers. Pattern Recognition, 43(1), 69-85. doi:10.1016/j.patcog.2009.05.013Vincent, E., Gribonval, R., & Fevotte, C. (2006). Performance measurement in blind audio source separation. IEEE Transactions on Audio, Speech and Language Processing, 14(4), 1462-1469. doi:10.1109/tsa.2005.858005Kullback, S., & Leibler, R. A. (1951). On Information and Sufficiency. The Annals of Mathematical Statistics, 22(1), 79-86. doi:10.1214/aoms/1177729694Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing, 13(4), 600-612. doi:10.1109/tip.2003.819861Raghavan, R. S. (1991). A model for spatially correlated radar clutter. IEEE Transactions on Aerospace and Electronic Systems, 27(2), 268-275. doi:10.1109/7.78302Hyvärinen, A., Hoyer, P. O., & Inki, M. (2001). Topographic Independent Component Analysis. Neural Computation, 13(7), 1527-1558. doi:10.1162/089976601750264992Salazar, A., Safont, G., & Vergara, L. (2011). Application of Independent Component Analysis for Evaluation of Ashlar Masonry Walls. Lecture Notes in Computer Science, 469-476. doi:10.1007/978-3-642-21498-1_5

Comportamento estocástico do algoritmo kernel least-mean-square

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-Graduação em Engenharia Elétrica.Algoritmos baseados em kernel têm-se tornado populares no processamento não-linear de sinais. O processamento não-linear aplicado sobre um sinal pode ser modelado como um processamento linear aplicado a um sinal transformado para um espaço de Hilbert com kernels reprodutivos (RKHS). A operação linear no espaço transformado pode ser implementada com baixa complexidade e pode ser melhor estudada e projetada. O algoritmo Kernel Least-Mean-Squares (KLMS) é um algoritmo popular em filtragem adaptativa não-linear devido à sua simplicidade e robustez. Implementações práticas desse algoritmo requerem um modelo de ordem finita do processamento não-linear, o que modifica o comportamento do algoritmo em relação ao LMS simplesmente mapeado para o RKHS. Essa modificação leva à necessidade de novos modelos analíticos para o comportamento do algoritmo. O desempenho do algoritmo é função do passo de convergência e dos parâmetros do kernel empregado. Este trabalho estuda o comportamento do KLMS em regimes transitório e permanente para entradas Gaussianas e um modelo de não-linearidade de ordem finita. Dois kernels são considerados; o Gaussiano e o Polinomial. Derivamos modelos analíticos recursivos para os comportamentos do vetor médio de erros nos coeficientes e do erro quadrático médio de estimação. As previsões do modelo mostram excelente acordo com simulações de Monte Carlo no transitório e no regime permanente. Isso permite a determinação explícita das condições para a estabilidade, e permite escolher os parâmetros do algoritmo a fim de obter um desempenho desejado. Exemplos de projeto são apresentados para o kernel Gaussiano e para o kernel Polinomial de segundo grau de forma a validar a análise teórica e ilustrar sua aplicação.Kernel-based algorithms have become popular in nonlinear signal processing. A nonlinear processing can be modeled as a linear processing applied to a signal transformed to a reproducing kernel Hilbert space (RKHS). The linear operation in the transformed space can be implemented with low computational complexity and can be more easily studied and designed. The Kernel Least-Mean-Squares (KLMS) is a popular algorithm in nonlinear adaptive filtering due to its simplicity and robustness. Practical implementations of this algorithm require a finite order model for the nonlinear processing. This modifies the algorithm behavior as compared to the LMS simply mapped to the RKHS. This modification leads to the need for new analytical models for the algorithm behavior. The algorithm behavior is a function of both the step size and the kernel parameters. This work studies the KLMS algorithm behavior in transient and in steady-state for Gaussian inputs and for a finite order nonlinearity model. Two kernels are considered; the Gaussian and the Polinomial. We derive analytical models for the behavior of both the mean weight error vector and the mean-square estimation error. The model predictions show excellent agreement with Monte Carlo simulations at both the transient and the steady-state. This allows the explicit determination of the stability limits and to design the algorithm parameters to obtain a desired performance. Design examples are presented for the Gaussian and for the second degree Polinomial kernels to validate the analysis and to illustrate its application