Global optimization for low-dimensional switching linear regression and bounded-error estimation

The paper provides global optimization algorithms for two particularly difficult nonconvex problems raised by hybrid system identification: switching linear regression and bounded-error estimation. While most works focus on local optimization heuristics without global optimality guarantees or with guarantees valid only under restrictive conditions, the proposed approach always yields a solution with a certificate of global optimality. This approach relies on a branch-and-bound strategy for which we devise lower bounds that can be efficiently computed. In order to obtain scalable algorithms with respect to the number of data, we directly optimize the model parameters in a continuous optimization setting without involving integer variables. Numerical experiments show that the proposed algorithms offer a higher accuracy than convex relaxations with a reasonable computational burden for hybrid system identification. In addition, we discuss how bounded-error estimation is related to robust estimation in the presence of outliers and exact recovery under sparse noise, for which we also obtain promising numerical results

Structural risk minimization for switched system identification

International audienceThis paper deals with the identification of hybrid dynamical systems that switch arbitrarily between modes. In particular, we focus on the critical issue of estimating the number of modes. A novel method inspired by model selection techniques in statistical learning is proposed. Specifically, the method implements the structural risk minimization principle, which relies on the minimization of an upper bound on the expected prediction error of the model. This so-called generalization error bound is first derived for static switched systems using Rademacher complexities. Then, it is extended to handle non independent observations from a single trajectory of a dynamical system. Finally, it is further tailored to the needs of model selection via a uniformization step. An illustrative example of the behavior of the method and its ability to recover the true number of modes is presented

On the complexity of piecewise affine system identification

International audienceThe paper provides results regarding the computational complexity of hybrid system identification. More precisely, we focus on the estimation of piecewise affine (PWA) maps from input-output data and analyze the complexity of computing a global minimizer of the error. Previous work showed that a global solution could be obtained for continuous PWA maps with a worst-case complexity exponential in the number of data. In this paper, we show how global optimality can be reached for a slightly more general class of possibly discontinuous PWA maps with a complexity only polynomial in the number of data, however with an exponential complexity with respect to the data dimension. This result is obtained via an analysis of the intrinsic classification subproblem of associating the data points to the different modes. In addition, we prove that the problem is NP-hard, and thus that the exponential complexity in the dimension is a natural expectation for any exact algorithm

Statistical Learning for Analysis of Networked Control Systems over Unknown Channels

Recent control trends are increasingly relying on communication networks and wireless channels to close the loop for Internet-of-Things applications. Traditionally these approaches are model-based, i.e., assuming a network or channel model they are focused on stability analysis and appropriate controller designs. However the availability of such wireless channel modeling is fundamentally challenging in practice as channels are typically unknown a priori and only available through data samples. In this work we aim to develop algorithms that rely on channel sample data to determine the stability and performance of networked control tasks. In this regard our work is the first to characterize the amount of channel modeling that is required to answer such a question. Specifically we examine how many channel data samples are required in order to answer with high confidence whether a given networked control system is stable or not. This analysis is based on the notion of sample complexity from the learning literature and is facilitated by concentration inequalities. Moreover we establish a direct relation between the sample complexity and the networked system stability margin, i.e., the underlying packet success rate of the channel and the spectral radius of the dynamics of the control system. This illustrates that it becomes impractical to verify stability under a large range of plant and channel configurations. We validate our theoretical results in numerical simulations

Realization of multi-input/multi-output switched linear systems from Markov parameters

This paper presents a four-stage algorithm for the realization of multi-input/multi-output (MIMO) switched linear systems (SLSs) from Markov parameters. In the first stage, a linear time-varying (LTV) realization that is topologically equivalent to the true SLS is derived from the Markov parameters assuming that the submodels have a common MacMillan degree and a mild condition on their dwell times holds. In the second stage, zero sets of LTV Hankel matrices where the realized system has a linear time-invariant (LTI) pulse response matching that of the original SLS are exploited to extract the submodels, up to arbitrary similarity transformations, by a clustering algorithm using a statistics that is invariant to similarity transformations. Recovery is shown to be complete if the dwell times are sufficiently long and some mild identifiability conditions are met. In the third stage, the switching sequence is estimated by three schemes. The first scheme is based on forward/backward corrections and works on the short segments. The second scheme matches Markov parameter estimates to the true parameters for LTV systems and works on the medium-to-long segments. The third scheme also matches Markov parameters, but for LTI systems only and works on the very short segments. In the fourth stage, the submodels estimated in Stage~2 are brought to a common basis by applying a novel basis transformation method which is necessary before performing output predictions to given inputs. A numerical example illustrates the properties of the realization algorithm. A key role in this algorithm is played by time-dependent switching sequences that partition the state-space according to time, unlike many other works in the literature in which partitioning is state and/or input dependent

Hybrid System Identification of Manual Tracking Submovements in Parkinson\u27s Disease

Seemingly smooth motions in manual tracking, (e.g., following a moving target with a joystick input) are actually sequences of submovements: short, open-loop motions that have been previously learned. In Parkinson\u27s disease, a neurodegenerative movement disorder, characterizations of motor performance can yield insight into underlying neurological mechanisms and therefore into potential treatment strategies. We focus on characterizing submovements through Hybrid System Identification, in which the dynamics of each submovement, the mode sequence and timing, and switching mechanisms are all unknown. We describe an initialization that provides a mode sequence and estimate of the dynamics of submovements, then apply hybrid optimization techniques based on embedding to solve a constrained nonlinear program. We also use the existing geometric approach for hybrid system identification to analyze our model and explain the deficits and advantages of each. These methods are applied to data gathered from subjects with Parkinson\u27s disease (on and off L-dopa medication) and from age-matched control subjects, and the results compared across groups demonstrating robust differences. Lastly, we develop a scheme to estimate the switching mechanism of the modeled hybrid system by using the principle of maximum margin separating hyperplane, which is a convex optimization problem, over the affine parameters describing the switching surface and provide a means o characterizing when too many or too few parameters are hypothesized to lie in the switching surface

A Sparsification Approach to Set Membership Identification of Switched Affine Systems

This paper addresses the problem of robust identification of a class of discrete-time affine hybrid systems, switched affine models, in a set membership framework. Given a finite collection of noisy input/output data and some minimal a priori information about the set of admissible plants, the objective is to identify a suitable set of affine models along with a switching sequence that can explain the available experimental information, while minimizing either the number of switches or subsystems. For the case where it is desired to minimize the number of switches, the key idea of the paper is to reduce this problem to a sparsification form, where the goal is to maximize sparsity of a suitably constructed vector sequence. Our main result shows that in the case of ℓ∞ bounded noise, this sparsification problem can be exactly solved via convex optimization. In the general case where the noise is only known to belong to a convex set N, the problem is generically NP-hard. However, as we show in the paper, efficient convex relaxations can be obtained by exploiting recent results on sparse signal recovery. Similarly, we present both a sparsification formulation and a convex relaxation for the (known to be NP hard) case where it is desired to minimize the number of subsystems. These results are illustrated using two non-trivial problems arising in computer vision applications: video-shot and dynamic texture segmentation