272 research outputs found
Outlier robust system identification: a Bayesian kernel-based approach
In this paper, we propose an outlier-robust regularized kernel-based method
for linear system identification. The unknown impulse response is modeled as a
zero-mean Gaussian process whose covariance (kernel) is given by the recently
proposed stable spline kernel, which encodes information on regularity and
exponential stability. To build robustness to outliers, we model the
measurement noise as realizations of independent Laplacian random variables.
The identification problem is cast in a Bayesian framework, and solved by a new
Markov Chain Monte Carlo (MCMC) scheme. In particular, exploiting the
representation of the Laplacian random variables as scale mixtures of
Gaussians, we design a Gibbs sampler which quickly converges to the target
distribution. Numerical simulations show a substantial improvement in the
accuracy of the estimates over state-of-the-art kernel-based methods.Comment: 5 figure
Bayesian kernel-based system identification with quantized output data
In this paper we introduce a novel method for linear system identification
with quantized output data. We model the impulse response as a zero-mean
Gaussian process whose covariance (kernel) is given by the recently proposed
stable spline kernel, which encodes information on regularity and exponential
stability. This serves as a starting point to cast our system identification
problem into a Bayesian framework. We employ Markov Chain Monte Carlo (MCMC)
methods to provide an estimate of the system. In particular, we show how to
design a Gibbs sampler which quickly converges to the target distribution.
Numerical simulations show a substantial improvement in the accuracy of the
estimates over state-of-the-art kernel-based methods when employed in
identification of systems with quantized data.Comment: Submitted to IFAC SysId 201
A Bayesian Approach to Sparse plus Low rank Network Identification
We consider the problem of modeling multivariate time series with
parsimonious dynamical models which can be represented as sparse dynamic
Bayesian networks with few latent nodes. This structure translates into a
sparse plus low rank model. In this paper, we propose a Gaussian regression
approach to identify such a model
Regularized Nonparametric Volterra Kernel Estimation
In this paper, the regularization approach introduced recently for
nonparametric estimation of linear systems is extended to the estimation of
nonlinear systems modelled as Volterra series. The kernels of order higher than
one, representing higher dimensional impulse responses in the series, are
considered to be realizations of multidimensional Gaussian processes. Based on
this, prior information about the structure of the Volterra kernel is
introduced via an appropriate penalization term in the least squares cost
function. It is shown that the proposed method is able to deliver accurate
estimates of the Volterra kernels even in the case of a small amount of data
points
Reweighted nuclear norm regularization: A SPARSEVA approach
The aim of this paper is to develop a method to estimate high order FIR and
ARX models using least squares with re-weighted nuclear norm regularization.
Typically, the choice of the tuning parameter in the reweighting scheme is
computationally expensive, hence we propose the use of the SPARSEVA (SPARSe
Estimation based on a VAlidation criterion) framework to overcome this problem.
Furthermore, we suggest the use of the prediction error criterion (PEC) to
select the tuning parameter in the SPARSEVA algorithm. Numerical examples
demonstrate the veracity of this method which has close ties with the
traditional technique of cross validation, but using much less computations.Comment: This paper is accepted and will be published in The Proceedings of
the 17th IFAC Symposium on System Identification (SYSID 2015), Beijing,
China, 201
Regularized system identification using orthonormal basis functions
Most of existing results on regularized system identification focus on
regularized impulse response estimation. Since the impulse response model is a
special case of orthonormal basis functions, it is interesting to consider if
it is possible to tackle the regularized system identification using more
compact orthonormal basis functions. In this paper, we explore two
possibilities. First, we construct reproducing kernel Hilbert space of impulse
responses by orthonormal basis functions and then use the induced reproducing
kernel for the regularized impulse response estimation. Second, we extend the
regularization method from impulse response estimation to the more general
orthonormal basis functions estimation. For both cases, the poles of the basis
functions are treated as hyperparameters and estimated by empirical Bayes
method. Then we further show that the former is a special case of the latter,
and more specifically, the former is equivalent to ridge regression of the
coefficients of the orthonormal basis functions.Comment: 6 pages, final submission of an contribution for European Control
Conference 2015, uploaded on March 20, 201
Maximum entropy properties of discrete-time first-order stable spline kernel
The first order stable spline (SS-1) kernel is used extensively in
regularized system identification. In particular, the stable spline estimator
models the impulse response as a zero-mean Gaussian process whose covariance is
given by the SS-1 kernel. In this paper, we discuss the maximum entropy
properties of this prior. In particular, we formulate the exact maximum entropy
problem solved by the SS-1 kernel without Gaussian and uniform sampling
assumptions. Under general sampling schemes, we also explicitly derive the
special structure underlying the SS-1 kernel (e.g. characterizing the
tridiagonal nature of its inverse), also giving to it a maximum entropy
covariance completion interpretation. Along the way similar maximum entropy
properties of the Wiener kernel are also given
Linear system identification using stable spline kernels and PLQ penalties
The classical approach to linear system identification is given by parametric
Prediction Error Methods (PEM). In this context, model complexity is often
unknown so that a model order selection step is needed to suitably trade-off
bias and variance. Recently, a different approach to linear system
identification has been introduced, where model order determination is avoided
by using a regularized least squares framework. In particular, the penalty term
on the impulse response is defined by so called stable spline kernels. They
embed information on regularity and BIBO stability, and depend on a small
number of parameters which can be estimated from data. In this paper, we
provide new nonsmooth formulations of the stable spline estimator. In
particular, we consider linear system identification problems in a very broad
context, where regularization functionals and data misfits can come from a rich
set of piecewise linear quadratic functions. Moreover, our anal- ysis includes
polyhedral inequality constraints on the unknown impulse response. For any
formulation in this class, we show that interior point methods can be used to
solve the system identification problem, with complexity O(n3)+O(mn2) in each
iteration, where n and m are the number of impulse response coefficients and
measurements, respectively. The usefulness of the framework is illustrated via
a numerical experiment where output measurements are contaminated by outliers.Comment: 8 pages, 2 figure
A new kernel-based approach to system identification with quantized output data
In this paper we introduce a novel method for linear system identification
with quantized output data. We model the impulse response as a zero-mean
Gaussian process whose covariance (kernel) is given by the recently proposed
stable spline kernel, which encodes information on regularity and exponential
stability. This serves as a starting point to cast our system identification
problem into a Bayesian framework. We employ Markov Chain Monte Carlo methods
to provide an estimate of the system. In particular, we design two methods
based on the so-called Gibbs sampler that allow also to estimate the kernel
hyperparameters by marginal likelihood maximization via the
expectation-maximization method. Numerical simulations show the effectiveness
of the proposed scheme, as compared to the state-of-the-art kernel-based
methods when these are employed in system identification with quantized data.Comment: 10 pages, 4 figure
- …