Gradient-based particle filter algorithm for an ARX model with nonlinear communication output

A stochastic gradient (SG)-based particle filter (SG-PF) algorithm is developed for an ARX model with nonlinear communication output in this paper. This ARX model consists of two submodels, one is a linear ARX model and the other is a nonlinear output model. The process outputs (outputs of the linear submodel) transmitted over a communication channel are unmeasurable, while the communication outputs (outputs of the nonlinear submodel) are available, and both of the twotype outputs are contaminated by white noises. Based on the rich input data and the available communication output data, a SG-PF algorithm is proposed to estimate the unknown process outputs and parameters of the ARX model. Furthermore, a direct weight optimization method and the Epanechnikov kernel method are extended to modify the particle filter when the measurement noise is a Gaussian noise with unknown variance and the measurement noise distribution is unknown. The simulation results demonstrate that the SG-PF algorithm is effective

Maximum Entropy Vector Kernels for MIMO system identification

Recent contributions have framed linear system identification as a nonparametric regularized inverse problem. Relying on $\ell_2$-type regularization which accounts for the stability and smoothness of the impulse response to be estimated, these approaches have been shown to be competitive w.r.t classical parametric methods. In this paper, adopting Maximum Entropy arguments, we derive a new $\ell_2$ penalty deriving from a vector-valued kernel; to do so we exploit the structure of the Hankel matrix, thus controlling at the same time complexity, measured by the McMillan degree, stability and smoothness of the identified models. As a special case we recover the nuclear norm penalty on the squared block Hankel matrix. In contrast with previous literature on reweighted nuclear norm penalties, our kernel is described by a small number of hyper-parameters, which are iteratively updated through marginal likelihood maximization; constraining the structure of the kernel acts as a (hyper)regularizer which helps controlling the effective degrees of freedom of our estimator. To optimize the marginal likelihood we adapt a Scaled Gradient Projection (SGP) algorithm which is proved to be significantly computationally cheaper than other first and second order off-the-shelf optimization methods. The paper also contains an extensive comparison with many state-of-the-art methods on several Monte-Carlo studies, which confirms the effectiveness of our procedure

Two iterative reweighted algorithms for systems contaminated by outliers

This study proposes two iterative reweighted (IRE) algorithms for systems whose data are contaminated by outliers. For the negative effect caused by the outliers, traditional least squares (LSs) and gradient descent (GD) algorithms cannot obtain unbiased estimates, while the variational Bayesian (VB) and expectation–maximization (EM) algorithms have the assumption that the prior knowledge of the outlier is available. To deal with these dilemmas, two IRE algorithms are developed. By assigning suitable weights for each dataset, unbiased parameter estimates can be obtained. In addition, the weights of the corrupted datasets become smaller and smaller with the increased number of iterations, and then, the contaminated data can be picked out from the datasets. The proposed algorithms do not require the prior knowledge of the outliers. Convergence analysis and numerical experiments show the effectiveness of the IRE algorithms

A comprehensive expectation identification framework for multirate time-delayed systems

The expectation maximization (EM) algorithm has been extensively used to solve system identification problems with hidden variables. It needs to calculate a derivative equation and perform a matrix inversion in the EM-M step. The equations related to the EM algorithm may be unsolvable for some complex nonlinear systems, and the matrix inversion has heavy computational costs for large-scale systems. This article provides two expectation-based algorithms with the aim of constructing a comprehensive expectation framework concerning different kinds of time-delayed systems: 1) for a small-scale linear system, the classical EM algorithm can quickly obtain the parameter and time-delay estimates; 2) for a complex nonlinear system with low order, the proposed expectation gradient descent algorithm can avoid derivative function calculation; 3) for a large-scale system, the proposed expectation multidirection algorithm does not require eigenvalue calculation and has less computational costs. These two algorithms are developed based on the gradient descent and multidirection methods. Under such an expectation framework, different kinds of models are identified on a case-by-case basis. The convergence analysis and simulation examples show the effectiveness of the algorithms

Biased compensation recursive least squares-based threshold algorithm for time-delay rational models via redundant rule

© 2017, Springer Science+Business Media B.V., part of Springer Nature. This paper develops a biased compensation recursive least squares-based threshold algorithm for a time-delay rational model. The time-delay rational model is transformed into an augmented model by using the redundant rule, and then, a recursive least squares algorithm is proposed to estimate the parameters of the augmented model. Since the output of the augmented model is correlated with the noise, a biased compensation method is derived to eliminate the bias of the parameter estimates. Furthermore, based on the structures of the augmented model parameter vector and the rational model parameter vector, the unknown time delay can be computed by using a threshold given in prior. A simulated example is used to illustrate the efficiency of the proposed algorithm

Robust standard gradient descent algorithm for ARX models using Aitken acceleration technique

A robust standard gradient descent (SGD) algorithm for ARX models using the Aitken acceleration method is developed. Considering that the SGD algorithm has slow convergence rates and is sensitive to the step size, a robust and accelerative SGD (RA-SGD) algorithm is derived. This algorithm is based on the Aitken acceleration method, and its convergence rate is improved from linear convergence to at least quadratic convergence in general. Furthermore, the RA-SGD algorithm is always convergent with no limitation of the step size. Both the convergence analysis and the simulation examples demonstrate that the presented algorithm is effective

Multidirection gradient iterative algorithm: A unified framework for gradient iterative and least squares algorithms

In this article, a multidirection-based gradient iterative (GI) algorithm for Hammerstein systems with irregular sampling data is proposed. The algorithm updates the parameter estimates using several orthogonal directions at each iteration. The convergence rate is significantly improved with an increasing number of directions. The convergence property and two simulation examples are provided to demonstrate the effectiveness of the proposed algorithm. In addition, the multidirection-based GI algorithm establishes a relationship between the traditional GI and least squares (LS) algorithms. Thus, our algorithm that combines the LS and GI algorithms constructs an identification framework for a significantly wider class of systems