A Sensitivity Matrix Methodology for Inverse Problem Formulation

We propose an algorithm to select parameter subset combinations that can be estimated using an ordinary least-squares (OLS) inverse problem formulation with a given data set. First, the algorithm selects the parameter combinations that correspond to sensitivity matrices with full rank. Second, the algorithm involves uncertainty quantification by using the inverse of the Fisher Information Matrix. Nominal values of parameters are used to construct synthetic data sets, and explore the effects of removing certain parameters from those to be estimated using OLS procedures. We quantify these effects in a score for a vector parameter defined using the norm of the vector of standard errors for components of estimates divided by the estimates. In some cases the method leads to reduction of the standard error for a parameter to less than 1% of the estimate

A unified framework for solving a general class of conditional and robust set-membership estimation problems

In this paper we present a unified framework for solving a general class of problems arising in the context of set-membership estimation/identification theory. More precisely, the paper aims at providing an original approach for the computation of optimal conditional and robust projection estimates in a nonlinear estimation setting where the operator relating the data and the parameter to be estimated is assumed to be a generic multivariate polynomial function and the uncertainties affecting the data are assumed to belong to semialgebraic sets. By noticing that the computation of both the conditional and the robust projection optimal estimators requires the solution to min-max optimization problems that share the same structure, we propose a unified two-stage approach based on semidefinite-relaxation techniques for solving such estimation problems. The key idea of the proposed procedure is to recognize that the optimal functional of the inner optimization problems can be approximated to any desired precision by a multivariate polynomial function by suitably exploiting recently proposed results in the field of parametric optimization. Two simulation examples are reported to show the effectiveness of the proposed approach.Comment: Accpeted for publication in the IEEE Transactions on Automatic Control (2014

Regularization and Bayesian Learning in Dynamical Systems: Past, Present and Future

Regularization and Bayesian methods for system identification have been repopularized in the recent years, and proved to be competitive w.r.t. classical parametric approaches. In this paper we shall make an attempt to illustrate how the use of regularization in system identification has evolved over the years, starting from the early contributions both in the Automatic Control as well as Econometrics and Statistics literature. In particular we shall discuss some fundamental issues such as compound estimation problems and exchangeability which play and important role in regularization and Bayesian approaches, as also illustrated in early publications in Statistics. The historical and foundational issues will be given more emphasis (and space), at the expense of the more recent developments which are only briefly discussed. The main reason for such a choice is that, while the recent literature is readily available, and surveys have already been published on the subject, in the author's opinion a clear link with past work had not been completely clarified.Comment: Plenary Presentation at the IFAC SYSID 2015. Submitted to Annual Reviews in Contro

Private Incremental Regression

Data is continuously generated by modern data sources, and a recent challenge in machine learning has been to develop techniques that perform well in an incremental (streaming) setting. In this paper, we investigate the problem of private machine learning, where as common in practice, the data is not given at once, but rather arrives incrementally over time. We introduce the problems of private incremental ERM and private incremental regression where the general goal is to always maintain a good empirical risk minimizer for the history observed under differential privacy. Our first contribution is a generic transformation of private batch ERM mechanisms into private incremental ERM mechanisms, based on a simple idea of invoking the private batch ERM procedure at some regular time intervals. We take this construction as a baseline for comparison. We then provide two mechanisms for the private incremental regression problem. Our first mechanism is based on privately constructing a noisy incremental gradient function, which is then used in a modified projected gradient procedure at every timestep. This mechanism has an excess empirical risk of $\approx\sqrt{d}$, where $d$ is the dimensionality of the data. While from the results of [Bassily et al. 2014] this bound is tight in the worst-case, we show that certain geometric properties of the input and constraint set can be used to derive significantly better results for certain interesting regression problems.Comment: To appear in PODS 201

Parametric high resolution techniques for radio astronomical imaging

The increased sensitivity of future radio telescopes will result in requirements for higher dynamic range within the image as well as better resolution and immunity to interference. In this paper we propose a new matrix formulation of the imaging equation in the cases of non co-planar arrays and polarimetric measurements. Then we improve our parametric imaging techniques in terms of resolution and estimation accuracy. This is done by enhancing both the MVDR parametric imaging, introducing alternative dirty images and by introducing better power estimates based on least squares, with positive semi-definite constraints. We also discuss the use of robust Capon beamforming and semi-definite programming for solving the self-calibration problem. Additionally we provide statistical analysis of the bias of the MVDR beamformer for the case of moving array, which serves as a first step in analyzing iterative approaches such as CLEAN and the techniques proposed in this paper. Finally we demonstrate a full deconvolution process based on the parametric imaging techniques and show its improved resolution and sensitivity compared to the CLEAN method.Comment: To appear in IEEE Journal of Selected Topics in Signal Processing, Special issue on Signal Processing for Astronomy and space research. 30 page