9,788 research outputs found
Global optimization for low-dimensional switching linear regression and bounded-error estimation
The paper provides global optimization algorithms for two particularly
difficult nonconvex problems raised by hybrid system identification: switching
linear regression and bounded-error estimation. While most works focus on local
optimization heuristics without global optimality guarantees or with guarantees
valid only under restrictive conditions, the proposed approach always yields a
solution with a certificate of global optimality. This approach relies on a
branch-and-bound strategy for which we devise lower bounds that can be
efficiently computed. In order to obtain scalable algorithms with respect to
the number of data, we directly optimize the model parameters in a continuous
optimization setting without involving integer variables. Numerical experiments
show that the proposed algorithms offer a higher accuracy than convex
relaxations with a reasonable computational burden for hybrid system
identification. In addition, we discuss how bounded-error estimation is related
to robust estimation in the presence of outliers and exact recovery under
sparse noise, for which we also obtain promising numerical results
Sequential Design for Optimal Stopping Problems
We propose a new approach to solve optimal stopping problems via simulation.
Working within the backward dynamic programming/Snell envelope framework, we
augment the methodology of Longstaff-Schwartz that focuses on approximating the
stopping strategy. Namely, we introduce adaptive generation of the stochastic
grids anchoring the simulated sample paths of the underlying state process.
This allows for active learning of the classifiers partitioning the state space
into the continuation and stopping regions. To this end, we examine sequential
design schemes that adaptively place new design points close to the stopping
boundaries. We then discuss dynamic regression algorithms that can implement
such recursive estimation and local refinement of the classifiers. The new
algorithm is illustrated with a variety of numerical experiments, showing that
an order of magnitude savings in terms of design size can be achieved. We also
compare with existing benchmarks in the context of pricing multi-dimensional
Bermudan options.Comment: 24 page
Error Bounds for Piecewise Smooth and Switching Regression
The paper deals with regression problems, in which the nonsmooth target is
assumed to switch between different operating modes. Specifically, piecewise
smooth (PWS) regression considers target functions switching deterministically
via a partition of the input space, while switching regression considers
arbitrary switching laws. The paper derives generalization error bounds in
these two settings by following the approach based on Rademacher complexities.
For PWS regression, our derivation involves a chaining argument and a
decomposition of the covering numbers of PWS classes in terms of the ones of
their component functions and the capacity of the classifier partitioning the
input space. This yields error bounds with a radical dependency on the number
of modes. For switching regression, the decomposition can be performed directly
at the level of the Rademacher complexities, which yields bounds with a linear
dependency on the number of modes. By using once more chaining and a
decomposition at the level of covering numbers, we show how to recover a
radical dependency. Examples of applications are given in particular for PWS
and swichting regression with linear and kernel-based component functions.Comment: This work has been submitted to the IEEE for possible publication.
Copyright may be transferred without notice,after which this version may no
longer be accessibl
Switching Regression Models and Causal Inference in the Presence of Discrete Latent Variables
Given a response and a vector of predictors,
we investigate the problem of inferring direct causes of among the vector
. Models for that use all of its causal covariates as predictors enjoy
the property of being invariant across different environments or interventional
settings. Given data from such environments, this property has been exploited
for causal discovery. Here, we extend this inference principle to situations in
which some (discrete-valued) direct causes of are unobserved. Such cases
naturally give rise to switching regression models. We provide sufficient
conditions for the existence, consistency and asymptotic normality of the MLE
in linear switching regression models with Gaussian noise, and construct a test
for the equality of such models. These results allow us to prove that the
proposed causal discovery method obtains asymptotic false discovery control
under mild conditions. We provide an algorithm, make available code, and test
our method on simulated data. It is robust against model violations and
outperforms state-of-the-art approaches. We further apply our method to a real
data set, where we show that it does not only output causal predictors, but
also a process-based clustering of data points, which could be of additional
interest to practitioners.Comment: 46 pages, 14 figures; real-world application added in Section 5.2;
additional numerical experiments added in the Appendix
Online Optimization with Memory and Competitive Control
This paper presents competitive algorithms for a novel class of online optimization problems with memory. We consider a setting where the learner seeks to minimize the sum of a hitting cost and a switching cost that depends on the previous p decisions. This setting generalizes Smoothed Online Convex Optimization. The proposed approach, Optimistic Regularized Online Balanced Descent, achieves a constant, dimension-free competitive ratio. Further, we show a connection between online optimization with memory and online control with adversarial disturbances. This connection, in turn, leads to a new constant-competitive policy for a rich class of online control problems
- …