408 research outputs found
Maximum Entropy Vector Kernels for MIMO system identification
Recent contributions have framed linear system identification as a
nonparametric regularized inverse problem. Relying on -type
regularization which accounts for the stability and smoothness of the impulse
response to be estimated, these approaches have been shown to be competitive
w.r.t classical parametric methods. In this paper, adopting Maximum Entropy
arguments, we derive a new penalty deriving from a vector-valued
kernel; to do so we exploit the structure of the Hankel matrix, thus
controlling at the same time complexity, measured by the McMillan degree,
stability and smoothness of the identified models. As a special case we recover
the nuclear norm penalty on the squared block Hankel matrix. In contrast with
previous literature on reweighted nuclear norm penalties, our kernel is
described by a small number of hyper-parameters, which are iteratively updated
through marginal likelihood maximization; constraining the structure of the
kernel acts as a (hyper)regularizer which helps controlling the effective
degrees of freedom of our estimator. To optimize the marginal likelihood we
adapt a Scaled Gradient Projection (SGP) algorithm which is proved to be
significantly computationally cheaper than other first and second order
off-the-shelf optimization methods. The paper also contains an extensive
comparison with many state-of-the-art methods on several Monte-Carlo studies,
which confirms the effectiveness of our procedure
Subspace System Identification via Weighted Nuclear Norm Optimization
We present a subspace system identification method based on weighted nuclear
norm approximation. The weight matrices used in the nuclear norm minimization
are the same weights as used in standard subspace identification methods. We
show that the inclusion of the weights improves the performance in terms of fit
on validation data. As a second benefit, the weights reduce the size of the
optimization problems that need to be solved. Experimental results from
randomly generated examples as well as from the Daisy benchmark collection are
reported. The key to an efficient implementation is the use of the alternating
direction method of multipliers to solve the optimization problem.Comment: Submitted to IEEE Conference on Decision and Contro
Rank-Sparsity Incoherence for Matrix Decomposition
Suppose we are given a matrix that is formed by adding an unknown sparse
matrix to an unknown low-rank matrix. Our goal is to decompose the given matrix
into its sparse and low-rank components. Such a problem arises in a number of
applications in model and system identification, and is NP-hard in general. In
this paper we consider a convex optimization formulation to splitting the
specified matrix into its components, by minimizing a linear combination of the
norm and the nuclear norm of the components. We develop a notion of
\emph{rank-sparsity incoherence}, expressed as an uncertainty principle between
the sparsity pattern of a matrix and its row and column spaces, and use it to
characterize both fundamental identifiability as well as (deterministic)
sufficient conditions for exact recovery. Our analysis is geometric in nature,
with the tangent spaces to the algebraic varieties of sparse and low-rank
matrices playing a prominent role. When the sparse and low-rank matrices are
drawn from certain natural random ensembles, we show that the sufficient
conditions for exact recovery are satisfied with high probability. We conclude
with simulation results on synthetic matrix decomposition problems
Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization
The affine rank minimization problem consists of finding a matrix of minimum
rank that satisfies a given system of linear equality constraints. Such
problems have appeared in the literature of a diverse set of fields including
system identification and control, Euclidean embedding, and collaborative
filtering. Although specific instances can often be solved with specialized
algorithms, the general affine rank minimization problem is NP-hard. In this
paper, we show that if a certain restricted isometry property holds for the
linear transformation defining the constraints, the minimum rank solution can
be recovered by solving a convex optimization problem, namely the minimization
of the nuclear norm over the given affine space. We present several random
ensembles of equations where the restricted isometry property holds with
overwhelming probability. The techniques used in our analysis have strong
parallels in the compressed sensing framework. We discuss how affine rank
minimization generalizes this pre-existing concept and outline a dictionary
relating concepts from cardinality minimization to those of rank minimization
- …