88 research outputs found
Global optimization for low-dimensional switching linear regression and bounded-error estimation
The paper provides global optimization algorithms for two particularly
difficult nonconvex problems raised by hybrid system identification: switching
linear regression and bounded-error estimation. While most works focus on local
optimization heuristics without global optimality guarantees or with guarantees
valid only under restrictive conditions, the proposed approach always yields a
solution with a certificate of global optimality. This approach relies on a
branch-and-bound strategy for which we devise lower bounds that can be
efficiently computed. In order to obtain scalable algorithms with respect to
the number of data, we directly optimize the model parameters in a continuous
optimization setting without involving integer variables. Numerical experiments
show that the proposed algorithms offer a higher accuracy than convex
relaxations with a reasonable computational burden for hybrid system
identification. In addition, we discuss how bounded-error estimation is related
to robust estimation in the presence of outliers and exact recovery under
sparse noise, for which we also obtain promising numerical results
Analysis of A Nonsmooth Optimization Approach to Robust Estimation
In this paper, we consider the problem of identifying a linear map from
measurements which are subject to intermittent and arbitarily large errors.
This is a fundamental problem in many estimation-related applications such as
fault detection, state estimation in lossy networks, hybrid system
identification, robust estimation, etc. The problem is hard because it exhibits
some intrinsic combinatorial features. Therefore, obtaining an effective
solution necessitates relaxations that are both solvable at a reasonable cost
and effective in the sense that they can return the true parameter vector. The
current paper discusses a nonsmooth convex optimization approach and provides a
new analysis of its behavior. In particular, it is shown that under appropriate
conditions on the data, an exact estimate can be recovered from data corrupted
by a large (even infinite) number of gross errors.Comment: 17 pages, 9 figure
Estimating the probability of success of a simple algorithm for switched linear regression
International audienceThis paper deals with the switched linear regression problem inherent in hybrid system identification. In particular, we discuss k-LinReg, a straightforward and easy to implement algorithm in the spirit of k-means for the nonconvex optimization problem at the core of switched linear regression, and focus on the question of its accuracy on large data sets and its ability to reach global optimality. To this end, we emphasize the relationship between the sample size and the probability of obtaining a local minimum close to the global one with a random initialization. This is achieved through the estimation of a model of the behavior of this probability with respect to the problem dimensions. This model can then be used to tune the number of restarts required to obtain a global solution with high probability. Experiments show that the model can accurately predict the probability of success and that, despite its simplicity, the resulting algorithm can outperform more complicated approaches in both speed and accuracy
On a class of optimization-based robust estimators
We consider in this paper the problem of estimating a parameter matrix from
observations which are affected by two types of noise components: (i) a sparse
noise sequence which, whenever nonzero can have arbitrarily large amplitude
(ii) and a dense and bounded noise sequence of "moderate" amount. This is
termed a robust regression problem. To tackle it, a quite general
optimization-based framework is proposed and analyzed. When only the sparse
noise is present, a sufficient bound is derived on the number of nonzero
elements in the sparse noise sequence that can be accommodated by the estimator
while still returning the true parameter matrix. While almost all the
restricted isometry-based bounds from the literature are not verifiable, our
bound can be easily computed through solving a convex optimization problem.
Moreover, empirical evidence tends to suggest that it is generally tight. If in
addition to the sparse noise sequence, the training data are affected by a
bounded dense noise, we derive an upper bound on the estimation error.Comment: To appear in IEEE Transactions on Automatic Contro
TAPAS: Tricks to Accelerate (encrypted) Prediction As a Service
Machine learning methods are widely used for a variety of prediction
problems. \emph{Prediction as a service} is a paradigm in which service
providers with technological expertise and computational resources may perform
predictions for clients. However, data privacy severely restricts the
applicability of such services, unless measures to keep client data private
(even from the service provider) are designed. Equally important is to minimize
the amount of computation and communication required between client and server.
Fully homomorphic encryption offers a possible way out, whereby clients may
encrypt their data, and on which the server may perform arithmetic
computations. The main drawback of using fully homomorphic encryption is the
amount of time required to evaluate large machine learning models on encrypted
data. We combine ideas from the machine learning literature, particularly work
on binarization and sparsification of neural networks, together with
algorithmic tools to speed-up and parallelize computation using encrypted data.Comment: Accepted at International Conference in Machine Learning (ICML), 201
On the complexity of piecewise affine system identification
International audienceThe paper provides results regarding the computational complexity of hybrid system identification. More precisely, we focus on the estimation of piecewise affine (PWA) maps from input-output data and analyze the complexity of computing a global minimizer of the error. Previous work showed that a global solution could be obtained for continuous PWA maps with a worst-case complexity exponential in the number of data. In this paper, we show how global optimality can be reached for a slightly more general class of possibly discontinuous PWA maps with a complexity only polynomial in the number of data, however with an exponential complexity with respect to the data dimension. This result is obtained via an analysis of the intrinsic classification subproblem of associating the data points to the different modes. In addition, we prove that the problem is NP-hard, and thus that the exponential complexity in the dimension is a natural expectation for any exact algorithm
Hybrid System Identification of Manual Tracking Submovements in Parkinson\u27s Disease
Seemingly smooth motions in manual tracking, (e.g., following a moving target with a joystick input) are actually sequences of submovements: short, open-loop motions that have been previously learned. In Parkinson\u27s disease, a neurodegenerative movement disorder, characterizations of motor performance can yield insight into underlying neurological mechanisms and therefore into potential treatment strategies. We focus on characterizing submovements through Hybrid System Identification, in which the dynamics of each submovement, the mode sequence and timing, and switching mechanisms are all unknown. We describe an initialization that provides a mode sequence and estimate of the dynamics of submovements, then apply hybrid optimization techniques based on embedding to solve a constrained nonlinear program. We also use the existing geometric approach for hybrid system identification to analyze our model and explain the deficits and advantages of each. These methods are applied to data gathered from subjects with Parkinson\u27s disease (on and off L-dopa medication) and from age-matched control subjects, and the results compared across groups demonstrating robust differences. Lastly, we develop a scheme to estimate the switching mechanism of the modeled hybrid system by using the principle of maximum margin separating hyperplane, which is a convex optimization problem, over the affine parameters describing the switching surface and provide a means o characterizing when too many or too few parameters are hypothesized to lie in the switching surface
- …