Discrete-time variance tracking with application to speech processing

Two new discrete-time algorithms are presented for tracking variance and reciprocal variance. The closed loop nature of the solutions to these problems makes this approach highly accurate and can be used recursively in real time. Since the Least-Mean Squares (LMS) method of parameter estimation requires an estimate of variance to compute the step size, this technique is well suited to applications such as speech processing and adaptive filtering

Sparse Volterra and Polynomial Regression Models: Recoverability and Estimation

Volterra and polynomial regression models play a major role in nonlinear system identification and inference tasks. Exciting applications ranging from neuroscience to genome-wide association analysis build on these models with the additional requirement of parsimony. This requirement has high interpretative value, but unfortunately cannot be met by least-squares based or kernel regression methods. To this end, compressed sampling (CS) approaches, already successful in linear regression settings, can offer a viable alternative. The viability of CS for sparse Volterra and polynomial models is the core theme of this work. A common sparse regression task is initially posed for the two models. Building on (weighted) Lasso-based schemes, an adaptive RLS-type algorithm is developed for sparse polynomial regressions. The identifiability of polynomial models is critically challenged by dimensionality. However, following the CS principle, when these models are sparse, they could be recovered by far fewer measurements. To quantify the sufficient number of measurements for a given level of sparsity, restricted isometry properties (RIP) are investigated in commonly met polynomial regression settings, generalizing known results for their linear counterparts. The merits of the novel (weighted) adaptive CS algorithms to sparse polynomial modeling are verified through synthetic as well as real data tests for genotype-phenotype analysis.Comment: 20 pages, to appear in IEEE Trans. on Signal Processin

Control of flexible joint robotic manipulator using tuning functions design

The goal of this thesis is to design the controller for a single arm manipulator having a flexible joint for the tracking problem in two different cases. A controller is designed for a deterministic case wherein the plant parameters are assumed to be known while another is designed for an adaptive case where all the plant parameters are assumed to be unknown. In general the tracking problem is; given a smooth reference trajectory, the end effector has to track the reference while maintaining the stability. It is assumed that only the output of the manipulator, which is the link angle, is available for measurement. Also without loss of generality, the fast dynamics, that is the dynamics of the driver side of the system are neglected for the sake of simplicity; In the first case, the design procedure adopted is called observer backstepping. Since the states of the system are unavailable for measurement, an observer is designed that estimates the system states. These estimates are fed to the controller which in turn produces the control input to the system; The second case employs a design procedure called tuning functions design. In this case, since the plant parameters are unknown, the observer designed in case one cannot be used for determining the state estimates. For this purpose, parameter update laws and filters are designed for estimation of plant parameters. The filters employed are k-filters. The k-filters and the parameter update laws are given as input to the controller, which generates the control input to the system; For both cases, the mathematical models are simulated using Matlab/Simulink, and the results are verified

Volterra and general polynomial related filtering

Journal ArticleThis paper presents a review of polynomial filtering and, in particular, of tlie truncated Volterra filters. Following the introduction of the general properties of such filters, issues such as eficieiit realizations, design, adaptive algoritlims and stability are discussed

Output-error LMS bilinear filters with stability monitoring

Journal ArticleABSTRACT This paper introduces output-error LMS bilinear filters with stability monitoring. Bilinear filters are recursive nonlinear systems that belong to the class of polynomial systems. Because of the feedback structure, such models are able to represent many nonlinear systems efficiently. However, the usefulness of adaptive bilinear filters is greatly restricted unless they are guaranteed to perform in a stable manner. A stability monitoring scheme is proposed to overcome the stability problem. The paper concludes with simulation results that demonstrate the usefulness of oiir technique

Algorithms and Data Structures for Multi-Adaptive Time-Stepping

Multi-adaptive Galerkin methods are extensions of the standard continuous and discontinuous Galerkin methods for the numerical solution of initial value problems for ordinary or partial differential equations. In particular, the multi-adaptive methods allow individual and adaptive time steps to be used for different components or in different regions of space. We present algorithms for efficient multi-adaptive time-stepping, including the recursive construction of time slabs and adaptive time step selection. We also present data structures for efficient storage and interpolation of the multi-adaptive solution. The efficiency of the proposed algorithms and data structures is demonstrated for a series of benchmark problems.Comment: ACM Transactions on Mathematical Software 35(3), 24 pages (2008

Distributed Recursive Least-Squares: Stability and Performance Analysis

The recursive least-squares (RLS) algorithm has well-documented merits for reducing complexity and storage requirements, when it comes to online estimation of stationary signals as well as for tracking slowly-varying nonstationary processes. In this paper, a distributed recursive least-squares (D-RLS) algorithm is developed for cooperative estimation using ad hoc wireless sensor networks. Distributed iterations are obtained by minimizing a separable reformulation of the exponentially-weighted least-squares cost, using the alternating-minimization algorithm. Sensors carry out reduced-complexity tasks locally, and exchange messages with one-hop neighbors to consent on the network-wide estimates adaptively. A steady-state mean-square error (MSE) performance analysis of D-RLS is conducted, by studying a stochastically-driven `averaged' system that approximates the D-RLS dynamics asymptotically in time. For sensor observations that are linearly related to the time-invariant parameter vector sought, the simplifying independence setting assumptions facilitate deriving accurate closed-form expressions for the MSE steady-state values. The problems of mean- and MSE-sense stability of D-RLS are also investigated, and easily-checkable sufficient conditions are derived under which a steady-state is attained. Without resorting to diminishing step-sizes which compromise the tracking ability of D-RLS, stability ensures that per sensor estimates hover inside a ball of finite radius centered at the true parameter vector, with high-probability, even when inter-sensor communication links are noisy. Interestingly, computer simulations demonstrate that the theoretical findings are accurate also in the pragmatic settings whereby sensors acquire temporally-correlated data.Comment: 30 pages, 4 figures, submitted to IEEE Transactions on Signal Processin

Stochastic Behavior Analysis of the Gaussian Kernel Least-Mean-Square Algorithm

The kernel least-mean-square (KLMS) algorithm is a popular algorithm in nonlinear adaptive filtering due to its simplicity and robustness. In kernel adaptive filters, the statistics of the input to the linear filter depends on the parameters of the kernel employed. Moreover, practical implementations require a finite nonlinearity model order. A Gaussian KLMS has two design parameters, the step size and the Gaussian kernel bandwidth. Thus, its design requires analytical models for the algorithm behavior as a function of these two parameters. This paper studies the steady-state behavior and the transient behavior of the Gaussian KLMS algorithm for Gaussian inputs and a finite order nonlinearity model. In particular, we derive recursive expressions for the mean-weight-error vector and the mean-square-error. The model predictions show excellent agreement with Monte Carlo simulations in transient and steady state. This allows the explicit analytical determination of stability limits, and gives opportunity to choose the algorithm parameters a priori in order to achieve prescribed convergence speed and quality of the estimate. Design examples are presented which validate the theoretical analysis and illustrates its application