Discrete wavelet transform-based RI adaptive algorithm for system identification

In this paper, we propose a new adaptive filtering algorithm for system identification. The algorithm is based on the recursive inverse (RI) adaptive algorithm which suffers from low convergence rates in some applications; i.e., the eigenvalue spread of the autocorrelation matrix is relatively high. The proposed algorithm applies discrete-wavelet transform (DWT) to the input signal which, in turn, helps to overcome the low convergence rate of the RI algorithm with relatively small step-size(s). Different scenarios has been investigated in different noise environments in system identification setting. Experiments demonstrate the advantages of the proposed DWT recursive inverse (DWT-RI) filter in terms of convergence rate and mean-square-error (MSE) compared to the RI, discrete cosine transform LMS (DCTLMS), discrete-wavelet transform LMS (DWT-LMS) and recursive-least-squares (RLS) algorithms under same conditions

The recursive Hessian sketch for adaptive filtering

We introduce in this paper the recursive Hessian sketch, a new adaptive filtering algorithm based on sketching the same exponentially weighted least squares problem solved by the recursive least squares algorithm. The algorithm maintains a number of sketches of the inverse autocorrelation matrix and recursively updates them at random intervals. These are in turn used to update the unknown filter estimate. The complexity of the proposed algorithm compares favorably to that of recursive least squares. The convergence properties of this algorithm are studied through extensive numerical experiments. With an appropriate choice or parameters, its convergence speed falls between that of least mean squares and recursive least squares adaptive filters, with less computations than the latter

Underdetermined-order recursive least-squares adaptive filtering: The concept and algorithms

Published versio

Large-scale wave-front reconstruction for adaptive optics systems by use of a recursive filtering algorithm

We propose a new recursive filtering algorithm for wave-front reconstruction in a large-scale adaptive optics system. An embedding step is used in this recursive filtering algorithm to permit fast methods to be used for wave-front reconstruction on an annular aperture. This embedding step can be used alone with a direct residual error updating procedure or used with the preconditioned conjugate-gradient method as a preconditioning step. We derive the Hudgin and Fried filters for spectral-domain filtering, using the eigenvalue decomposition method. Using Monte Carlo simulations, we compare the performance of discrete Fourier transform domain filtering, discrete cosine transform domain filtering, multigrid, and alternative-direction-implicit methods in the embedding step of the recursive filtering algorithm. We also simulate the performance of this recursive filtering in a closed-loop adaptive optics system

A fast algorithm for LR-2 factorization of Toeplitz matrices

In this paper a new order recursive algorithm for the efficient −1 factorization of Toeplitz matrices is described. The proposed algorithm can be seen as a fast modified Gram-Schmidt method which recursively computes the orthonormal columns i, i = 1,2, …,p, of , as well as the elements of R−1, of a Toeplitz matrix with dimensions L × p. The factor estimation requires 8Lp MADS (multiplications and divisions). Matrix −1 is subsequently estimated using 3p2 MADS. A faster algorithm, based on a mixed and −1 updating scheme, is also derived. It requires 7Lp + 3.5p2 MADS. The algorithm can be efficiently applied to batch least squares FIR filtering and system identification. When determination of the optimal filter is the desired task it can be utilized to compute the least squares filter in an order recursive way. The algorithm operates directly on the experimental data, overcoming the need for covariance estimates. An orthogonalized version of the proposed −1 algorithm is derived. Matlab code implementing the algorithm is also supplied

A Stochastic Majorize-Minimize Subspace Algorithm for Online Penalized Least Squares Estimation

Stochastic approximation techniques play an important role in solving many problems encountered in machine learning or adaptive signal processing. In these contexts, the statistics of the data are often unknown a priori or their direct computation is too intensive, and they have thus to be estimated online from the observed signals. For batch optimization of an objective function being the sum of a data fidelity term and a penalization (e.g. a sparsity promoting function), Majorize-Minimize (MM) methods have recently attracted much interest since they are fast, highly flexible, and effective in ensuring convergence. The goal of this paper is to show how these methods can be successfully extended to the case when the data fidelity term corresponds to a least squares criterion and the cost function is replaced by a sequence of stochastic approximations of it. In this context, we propose an online version of an MM subspace algorithm and we study its convergence by using suitable probabilistic tools. Simulation results illustrate the good practical performance of the proposed algorithm associated with a memory gradient subspace, when applied to both non-adaptive and adaptive filter identification problems