4,018 research outputs found
Identification of Stochastic Wiener Systems using Indirect Inference
We study identification of stochastic Wiener dynamic systems using so-called
indirect inference. The main idea is to first fit an auxiliary model to the
observed data and then in a second step, often by simulation, fit a more
structured model to the estimated auxiliary model. This two-step procedure can
be used when the direct maximum-likelihood estimate is difficult or intractable
to compute. One such example is the identification of stochastic Wiener
systems, i.e.,~linear dynamic systems with process noise where the output is
measured using a non-linear sensor with additive measurement noise. It is in
principle possible to evaluate the log-likelihood cost function using numerical
integration, but the corresponding optimization problem can be quite intricate.
This motivates studying consistent, but sub-optimal, identification methods for
stochastic Wiener systems. We will consider indirect inference using the best
linear approximation as an auxiliary model. We show that the key to obtain a
reliable estimate is to use uncertainty weighting when fitting the stochastic
Wiener model to the auxiliary model estimate. The main technical contribution
of this paper is the corresponding asymptotic variance analysis. A numerical
evaluation is presented based on a first-order finite impulse response system
with a cubic non-linearity, for which certain illustrative analytic properties
are derived.Comment: The 17th IFAC Symposium on System Identification, SYSID 2015,
Beijing, China, October 19-21, 201
Sparsity-Cognizant Total Least-Squares for Perturbed Compressive Sampling
Solving linear regression problems based on the total least-squares (TLS)
criterion has well-documented merits in various applications, where
perturbations appear both in the data vector as well as in the regression
matrix. However, existing TLS approaches do not account for sparsity possibly
present in the unknown vector of regression coefficients. On the other hand,
sparsity is the key attribute exploited by modern compressive sampling and
variable selection approaches to linear regression, which include noise in the
data, but do not account for perturbations in the regression matrix. The
present paper fills this gap by formulating and solving TLS optimization
problems under sparsity constraints. Near-optimum and reduced-complexity
suboptimum sparse (S-) TLS algorithms are developed to address the perturbed
compressive sampling (and the related dictionary learning) challenge, when
there is a mismatch between the true and adopted bases over which the unknown
vector is sparse. The novel S-TLS schemes also allow for perturbations in the
regression matrix of the least-absolute selection and shrinkage selection
operator (Lasso), and endow TLS approaches with ability to cope with sparse,
under-determined "errors-in-variables" models. Interesting generalizations can
further exploit prior knowledge on the perturbations to obtain novel weighted
and structured S-TLS solvers. Analysis and simulations demonstrate the
practical impact of S-TLS in calibrating the mismatch effects of contemporary
grid-based approaches to cognitive radio sensing, and robust
direction-of-arrival estimation using antenna arrays.Comment: 30 pages, 10 figures, submitted to IEEE Transactions on Signal
Processin
Recommended from our members
Identification of nonlinear interconnected systems
This thesis was submitted for the degree of Master of Philosophy and awarded by Brunel University.In this work we address the problem of identifying a discrete-time nonlinear system composed of a linear dynamical system connected to a static nonlinear component. We use linear fractional representation to provide a united framework for the identification of two classes of such systems. The first class consists of discrete-time systems consists of a linear time invariant system connected to a continuous nonlinear static component. The identification problem of estimating the unknown parameters of the linear system and simultaneously fitting a math order spline to the nonlinear data is addressed. A simple and tractable algorithm based on the separable least squares method is proposed for estimating the parameters of the linear
and the nonlinear components. We also provide a sufficient condition on data for consistency of the identification algorithm. Numerical examples illustrate the performance of the algorithm. Further, we examine a second class of systems that may involve a nonlinear static element of a more complex structure. The nonlinearity may not be continuous and is approximated by piecewise a±ne maps defined on different convex polyhedra, which are defined by linear
combinations of lagged inputs and outputs. An iterative identification procedure is proposed, which alternates the estimation of the linear and the nonlinear subsystems. Standard identification techniques are applied to the linear subsystem, whereas recently developed piecewise affine system identification techniques are employed for the estimation of the nonlinear component. Numerical examples show that the proposed procedure is able to successfully profit
from the knowledge of the interconnection structure, in comparison with a direct black box identification of the piecewise a±ne system.Funding was obtained as a Marie Curie Early Stage Researcher Training fellowship, under the NET-ACE project (MEST-CT-2004-6724)
Exploiting structure in piecewise affine identification of LFT systems
Identification of interconnected systems is a challenging problem in which it is crucial to exploit the available knowledge about the interconnection structure. In this paper, identification of discrete-time nonlinear systems composed by interconnected linear
and nonlinear systems, is addressed. An iterative identification procedure is proposed, which alternates the estimation of the linear and the nonlinear components. Standard identification techniques are applied to the linear subsystem, whereas recently developed piecewise affine (PWA) identification techniques are employed for modelling the nonlinearity. A numerical
example analyzes the benefits of the proposed structure-exploiting identification algorithm compared to applying black-box PWA identification techniques to the overall system
Hybrid Deterministic-Stochastic Methods for Data Fitting
Many structured data-fitting applications require the solution of an
optimization problem involving a sum over a potentially large number of
measurements. Incremental gradient algorithms offer inexpensive iterations by
sampling a subset of the terms in the sum. These methods can make great
progress initially, but often slow as they approach a solution. In contrast,
full-gradient methods achieve steady convergence at the expense of evaluating
the full objective and gradient on each iteration. We explore hybrid methods
that exhibit the benefits of both approaches. Rate-of-convergence analysis
shows that by controlling the sample size in an incremental gradient algorithm,
it is possible to maintain the steady convergence rates of full-gradient
methods. We detail a practical quasi-Newton implementation based on this
approach. Numerical experiments illustrate its potential benefits.Comment: 26 pages. Revised proofs of Theorems 2.6 and 3.1, results unchange
Let's Make Block Coordinate Descent Go Fast: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence
Block coordinate descent (BCD) methods are widely-used for large-scale
numerical optimization because of their cheap iteration costs, low memory
requirements, amenability to parallelization, and ability to exploit problem
structure. Three main algorithmic choices influence the performance of BCD
methods: the block partitioning strategy, the block selection rule, and the
block update rule. In this paper we explore all three of these building blocks
and propose variations for each that can lead to significantly faster BCD
methods. We (i) propose new greedy block-selection strategies that guarantee
more progress per iteration than the Gauss-Southwell rule; (ii) explore
practical issues like how to implement the new rules when using "variable"
blocks; (iii) explore the use of message-passing to compute matrix or Newton
updates efficiently on huge blocks for problems with a sparse dependency
between variables; and (iv) consider optimal active manifold identification,
which leads to bounds on the "active set complexity" of BCD methods and leads
to superlinear convergence for certain problems with sparse solutions (and in
some cases finite termination at an optimal solution). We support all of our
findings with numerical results for the classic machine learning problems of
least squares, logistic regression, multi-class logistic regression, label
propagation, and L1-regularization
- …