Underdetermined-order recursive least-squares adaptive filtering: The concept and algorithms

Published versio

Low-complexity RLS algorithms using dichotomous coordinate descent iterations

In this paper, we derive low-complexity recursive least squares (RLS) adaptive filtering algorithms. We express the RLS problem in terms of auxiliary normal equations with respect to increments of the filter weights and apply this approach to the exponentially weighted and sliding window cases to derive new RLS techniques. For solving the auxiliary equations, line search methods are used. We first consider conjugate gradient iterations with a complexity of O(N-2) operations per sample; N being the number of the filter weights. To reduce the complexity and make the algorithms more suitable for finite precision implementation, we propose a new dichotomous coordinate descent (DCD) algorithm and apply it to the auxiliary equations. This results in a transversal RLS adaptive filter with complexity as low as 3N multiplications per sample, which is only slightly higher than the complexity of the least mean squares (LMS) algorithm (2N multiplications). Simulations are used to compare the performance of the proposed algorithms against the classical RLS and known advanced adaptive algorithms. Fixed-point FPGA implementation of the proposed DCD-based RLS algorithm is also discussed and results of such implementation are presented

Sequential Design for Optimal Stopping Problems

We propose a new approach to solve optimal stopping problems via simulation. Working within the backward dynamic programming/Snell envelope framework, we augment the methodology of Longstaff-Schwartz that focuses on approximating the stopping strategy. Namely, we introduce adaptive generation of the stochastic grids anchoring the simulated sample paths of the underlying state process. This allows for active learning of the classifiers partitioning the state space into the continuation and stopping regions. To this end, we examine sequential design schemes that adaptively place new design points close to the stopping boundaries. We then discuss dynamic regression algorithms that can implement such recursive estimation and local refinement of the classifiers. The new algorithm is illustrated with a variety of numerical experiments, showing that an order of magnitude savings in terms of design size can be achieved. We also compare with existing benchmarks in the context of pricing multi-dimensional Bermudan options.Comment: 24 page

Robust recursive eigendecomposition and subspace-based algorithms with application to fault detection in wireless sensor networks

The principal component analysis (PCA) is a valuable tool in multivariate statistics, and it is an effective method for fault detection in wireless sensor networks (WSNs) and other related applications. However, its online implementation requires the computation of eigendecomposition (ED) or singular value decomposition. To reduce the arithmetic complexity, we propose an efficient fault detection approach using the subspace tracking concept. In particular, two new robust subspace tracking algorithms are developed, namely, the robust orthonormal projection approximation subspace tracking (OPAST) with rank-1 modification and the robust OPAST with deflation. Both methods rely on robust M-estimate-based recursive covariance estimate to improve the robustness against the effect of faulty samples, and they offer different tradeoff between fault detection accuracy and arithmetic complexity. Since only the ED in the major subspace is computed, their arithmetic complexities are much lower than those of other conventional PCA-based algorithms. Furthermore, we propose new robust T 2 score and SPE detection criteria with recursive update formulas to improve the robustness over their conventional counterparts and to facilitate online implementation for the proposed robust subspace ED and tracking algorithms. Computer simulation and experimental results on WSN data show that the proposed fault detection approach, which combines the aforementioned robust subspace tracking algorithms with the robust detection criteria, is able to achieve better performance than other conventional approaches. Hence, it serves as an attractive alternative to other conventional approaches to fault detection in WSNs and other related applications because of its low complexity, efficient recursive implementation, and good performance. © 2012 IEEE.published_or_final_versio

ESTIMATION OF VEHICLE MASS AND ROAD GRADE

This thesis describes development of a real-time-implementable algorithm for simultaneous estimation of a heavy vehicle\u27s mass and time-varying road grade and its verification with experimental data. Accurate estimate of a heavy vehicle\u27s mass is critical in several vehicle control functions such as in transmission and stability control. The goal is to utilize the standard signals on a vehicle in a model-based estimation strategy, as opposed to a more costly sensor-based approach. The challenge is that unknown road grade complicates model-based estimation of vehicle mass and therefore the time-varying grade should be estimated simultaneously. In addition an estimate of road grade may be used as a feedforward input to transmission control and cruise control systems enhancing their responsiveness. The vehicle longitudinal dynamics model (F=ma) forms the core of this model-based approach. Mathematically this is a single equation with one unknown parameter (mass) and one time-varying input disturbance (grade). The goal is to estimate the constant parameter and time-varying grade by using engine torque and speed, vehicle speed and transmission state. The problem is fundamentally difficult because of i) variation of grade over time ii) lack of ``rich\u27\u27 data during most of vehicle\u27s cruise time, iii) uncertainty about available traction force during gear-shift periods and braking, and iv) low signal-to-noise ratio for vehicle acceleration signal. We have tested two independent estimation schemes using experimental data sets provided by Eaton Corporation. The first algorithm uses recursive least square with two forgetting factors for simultaneous estimation of mass and grade. The second algorithm is a two-stage scheme which cascades a Lyapunov-based nonlinear estimator next to a recursive least square scheme. These algorithms were conceived in our group in the past; however they needed modification and refinements for robust real-time implementation. After these refinements, the modified algorithms are capable of generating estimates for mass and time-varying road grade which are more accurate in realistic scenarios and for most part of the vehicle run. More specifically we are able to generate very accurate estimates of road grade, when the clutch is fully engaged and we have proposed fixes that improve the quality of estimates even during periods of gear change. Provided persistence of excitations we are able to generate accurate estimates of mass which in turn improves the quality of grade estimate. It is important to robustify initialization of algorithm 1 further which is now sensitive to an initial batch size; a task listed in the future work. Algorithm 2 does not rely on an initial batch and therefore is expected to be adopted as the preferred approach for implementation

The wavelet-NARMAX representation : a hybrid model structure combining polynomial models with multiresolution wavelet decompositions

A new hybrid model structure combing polynomial models with multiresolution wavelet decompositions is introduced for nonlinear system identification. Polynomial models play an important role in approximation theory, and have been extensively used in linear and nonlinear system identification. Wavelet decompositions, in which the basis functions have the property of localization in both time and frequency, outperform many other approximation schemes and offer a flexible solution for approximating arbitrary functions. Although wavelet representations can approximate even severe nonlinearities in a given signal very well, the advantage of these representations can be lost when wavelets are used to capture linear or low-order nonlinear behaviour in a signal. In order to sufficiently utilise the global property of polynomials and the local property of wavelet representations simultaneously, in this study polynomial models and wavelet decompositions are combined together in a parallel structure to represent nonlinear input-output systems. As a special form of the NARMAX model, this hybrid model structure will be referred to as the WAvelet-NARMAX model, or simply WANARMAX. Generally, such a WANARMAX representation for an input-output system might involve a large number of basis functions and therefore a great number of model terms. Experience reveals that only a small number of these model terms are significant to the system output. A new fast orthogonal least squares algorithm, called the matching pursuit orthogonal least squares (MPOLS) algorithm, is also introduced in this study to determine which terms should be included in the final model