32 research outputs found
Finite Time Identification in Unstable Linear Systems
Identification of the parameters of stable linear dynamical systems is a
well-studied problem in the literature, both in the low and high-dimensional
settings. However, there are hardly any results for the unstable case,
especially regarding finite time bounds. For this setting, classical results on
least-squares estimation of the dynamics parameters are not applicable and
therefore new concepts and technical approaches need to be developed to address
the issue. Unstable linear systems arise in key real applications in control
theory, econometrics, and finance. This study establishes finite time bounds
for the identification error of the least-squares estimates for a fairly large
class of heavy-tailed noise distributions, and transition matrices of such
systems. The results relate the time length (samples) required for estimation
to a function of the problem dimension and key characteristics of the true
underlying transition matrix and the noise distribution. To establish them,
appropriate concentration inequalities for random matrices and for sequences of
martingale differences are leveraged
Randomized Algorithms for Data-Driven Stabilization of Stochastic Linear Systems
Data-driven control strategies for dynamical systems with unknown parameters
are popular in theory and applications. An essential problem is to prevent
stochastic linear systems becoming destabilized, due to the uncertainty of the
decision-maker about the dynamical parameter. Two randomized algorithms are
proposed for this problem, but the performance is not sufficiently
investigated. Further, the effect of key parameters of the algorithms such as
the magnitude and the frequency of applying the randomizations is not currently
available. This work studies the stabilization speed and the failure
probability of data-driven procedures. We provide numerical analyses for the
performance of two methods: stochastic feedback, and stochastic parameter. The
presented results imply that as long as the number of statistically independent
randomizations is not too small, fast stabilization is guaranteed
Sample Complexity Lower Bounds for Linear System Identification
This paper establishes problem-specific sample complexity lower bounds for
linear system identification problems. The sample complexity is defined in the
PAC framework: it corresponds to the time it takes to identify the system
parameters with prescribed accuracy and confidence levels. By problem-specific,
we mean that the lower bound explicitly depends on the system to be identified
(which contrasts with minimax lower bounds), and hence really captures the
identification hardness specific to the system. We consider both uncontrolled
and controlled systems. For uncontrolled systems, the lower bounds are valid
for any linear system, stable or not, and only depend of the system finite-time
controllability gramian. A simplified lower bound depending on the spectrum of
the system only is also derived. In view of recent finitetime analysis of
classical estimation methods (e.g. ordinary least squares), our sample
complexity lower bounds are tight for many systems. For controlled systems, our
lower bounds are not as explicit as in the case of uncontrolled systems, but
could well provide interesting insights into the design of control policy with
minimal sample complexity
Finite-time Identification of Stable Linear Systems: Optimality of the Least-Squares Estimator
We present a new finite-time analysis of the estimation error of the Ordinary
Least Squares (OLS) estimator for stable linear time-invariant systems. We
characterize the number of observed samples (the length of the observed
trajectory) sufficient for the OLS estimator to be -PAC,
i.e., to yield an estimation error less than with probability at
least . We show that this number matches existing sample complexity
lower bounds [1,2] up to universal multiplicative factors (independent of
( and of the system). This paper hence establishes the
optimality of the OLS estimator for stable systems, a result conjectured in
[1]. Our analysis of the performance of the OLS estimator is simpler, sharper,
and easier to interpret than existing analyses. It relies on new concentration
results for the covariates matrix
Finite Time Adaptive Stabilization of LQ Systems
Stabilization of linear systems with unknown dynamics is a canonical problem
in adaptive control. Since the lack of knowledge of system parameters can cause
it to become destabilized, an adaptive stabilization procedure is needed prior
to regulation. Therefore, the adaptive stabilization needs to be completed in
finite time. In order to achieve this goal, asymptotic approaches are not very
helpful. There are only a few existing non-asymptotic results and a full
treatment of the problem is not currently available.
In this work, leveraging the novel method of random linear feedbacks, we
establish high probability guarantees for finite time stabilization. Our
results hold for remarkably general settings because we carefully choose a
minimal set of assumptions. These include stabilizability of the underlying
system and restricting the degree of heaviness of the noise distribution. To
derive our results, we also introduce a number of new concepts and technical
tools to address regularity and instability of the closed-loop matrix.Comment: arXiv admin note: substantial text overlap with arXiv:1711.0723
Finite Sample Analysis of Stochastic System Identification
In this paper, we analyze the finite sample complexity of stochastic system
identification using modern tools from machine learning and statistics. An
unknown discrete-time linear system evolves over time under Gaussian noise
without external inputs. The objective is to recover the system parameters as
well as the Kalman filter gain, given a single trajectory of output
measurements over a finite horizon of length . Based on a subspace
identification algorithm and a finite number of output samples, we provide
non-asymptotic high-probability upper bounds for the system parameter
estimation errors. Our analysis uses recent results from random matrix theory,
self-normalized martingales and SVD robustness, in order to show that with high
probability the estimation errors decrease with a rate of . Our
non-asymptotic bounds not only agree with classical asymptotic results, but are
also valid even when the system is marginally stable.Comment: Under revie
Input Perturbations for Adaptive Control and Learning
This paper studies adaptive algorithms for simultaneous regulation (i.e.,
control) and estimation (i.e., learning) of Multiple Input Multiple Output
(MIMO) linear dynamical systems. It proposes practical, easy to implement
control policies based on perturbations of input signals. Such policies are
shown to achieve a worst-case regret that scales as the square-root of the time
horizon, and holds uniformly over time. Further, it discusses specific settings
where such greedy policies attain the information theoretic lower bound of
logarithmic regret. To establish the results, recent advances on
self-normalized martingales together with a novel method of policy
decomposition are leveraged
Optimism-Based Adaptive Regulation of Linear-Quadratic Systems
The main challenge for adaptive regulation of linear-quadratic systems is the
trade-off between identification and control. An adaptive policy needs to
address both the estimation of unknown dynamics parameters (exploration), as
well as the regulation of the underlying system (exploitation). To this end,
optimism-based methods which bias the identification in favor of optimistic
approximations of the true parameter are employed in the literature. A number
of asymptotic results have been established, but their finite time counterparts
are few, with important restrictions.
This study establishes results for the worst-case regret of optimism-based
adaptive policies. The presented high probability upper bounds are optimal up
to logarithmic factors. The non-asymptotic analysis of this work requires very
mild assumptions; (i) stabilizability of the system's dynamics, and (ii)
limiting the degree of heaviness of the noise distribution. To establish such
bounds, certain novel techniques are developed to comprehensively address the
probabilistic behavior of dependent random matrices with heavy-tailed
distributions.Comment: 28 page
Certainty Equivalence is Efficient for Linear Quadratic Control
We study the performance of the certainty equivalent controller on Linear
Quadratic (LQ) control problems with unknown transition dynamics. We show that
for both the fully and partially observed settings, the sub-optimality gap
between the cost incurred by playing the certainty equivalent controller on the
true system and the cost incurred by using the optimal LQ controller enjoys a
fast statistical rate, scaling as the square of the parameter error. To the
best of our knowledge, our result is the first sub-optimality guarantee in the
partially observed Linear Quadratic Gaussian (LQG) setting. Furthermore, in the
fully observed Linear Quadratic Regulator (LQR), our result improves upon
recent work by Dean et al. (2017), who present an algorithm achieving a
sub-optimality gap linear in the parameter error. A key part of our analysis
relies on perturbation bounds for discrete Riccati equations. We provide two
new perturbation bounds, one that expands on an existing result from
Konstantinov et al. (1993), and another based on a new elementary proof
strategy.Comment: In the current version we extended our analysis to the case of
partially observable systems, i.e. we provided a suboptimality analysis for
the Linear Quadratic Gaussian (LQG) settin
On Adaptive Linear-Quadratic Regulators
Performance of adaptive control policies is assessed through the regret with
respect to the optimal regulator, which reflects the increase in the operating
cost due to uncertainty about the dynamics parameters. However, available
results in the literature do not provide a quantitative characterization of the
effect of the unknown parameters on the regret. Further, there are problems
regarding the efficient implementation of some of the existing adaptive
policies. Finally, results regarding the accuracy with which the system's
parameters are identified are scarce and rather incomplete.
This study aims to comprehensively address these three issues. First, by
introducing a novel decomposition of adaptive policies, we establish a sharp
expression for the regret of an arbitrary policy in terms of the deviations
from the optimal regulator. Second, we show that adaptive policies based on
slight modifications of the Certainty Equivalence scheme are efficient.
Specifically, we establish a regret of (nearly) square-root rate for two
families of randomized adaptive policies. The presented regret bounds are
obtained by using anti-concentration results on the random matrices employed
for randomizing the estimates of the unknown parameters. Moreover, we study the
minimal additional information on dynamics matrices that using them the regret
will become of logarithmic order. Finally, the rates at which the unknown
parameters of the system are being identified are presented