Check Yourself Before You WREK Yourself: Unpacking and Generalizing Randomized Extended Kaczmarz

Linear systems are fundamental in many areas of science and engineering. With the advent of computers there now exist extremely large linear systems that we are interested in. Such linear systems lend themselves to iterative methods. One such method is the family of algorithms called Randomized Kaczmarz methods.Among this family, there exists a Randomized Kaczmarz variant called RandomizedExtended Kaczmarz which solves for least squares solutions in inconsistent linear systems. Among Kaczmarz variants, Randomized Extended Kaczmarz is unique in that it modifies input system in a special way to solve for the least squares solution. In this work we unpack the geometry underlying Randomized Extended Kaczmarz(REK) by uniting proofs by Zouzias and Freris (2013) and Du (2018), leading to more insight about why REK works. We also provide novel proofs showing: that REK will converge with an alternative sequence of z updates, and giving a closed form for REK’s original z updates. Lastly we have done some work generalizing the ideas behind REK and QuantileRK (Haddock et al., 2020) to lay foundations for a new Randomized Kaczmarz variant called Weighted Randomized Extended Kaczmarz (WREK) which aim to solve weighted least squares problems with dynamic reweightings

Penalized likelihood estimation and iterative kalman smoothing for non-gaussian dynamic regression models

Dynamic regression or state space models provide a flexible framework for analyzing non-Gaussian time series and longitudinal data, covering for example models for discrete longitudinal observations. As for non-Gaussian random coefficient models, a direct Bayesian approach leads to numerical integration problems, often intractable for more complicated data sets. Recent Markov chain Monte Carlo methods avoid this by repeated sampling from approximative posterior distributions, but there are still open questions about sampling schemes and convergence. In this article we consider simpler methods of inference based on posterior modes or, equivalently, maximum penalized likelihood estimation. From the latter point of view, the approach can also be interpreted as a nonparametric method for smoothing time-varying coefficients. Efficient smoothing algorithms are obtained by iteration of common linear Kalman filtering and smoothing, in the same way as estimation in generalized linear models with fixed effects can be performed by iteratively weighted least squares estimation. The algorithm can be combined with an EM-type method or cross-validation to estimate unknown hyper- or smoothing parameters. The approach is illustrated by applications to a binary time series and a multicategorical longitudinal data set

On the interpretation and identification of dynamic Takagi-Sugenofuzzy models

Dynamic Takagi-Sugeno fuzzy models are not always easy to interpret, in particular when they are identified from experimental data. It is shown that there exists a close relationship between dynamic Takagi-Sugeno fuzzy models and dynamic linearization when using affine local model structures, which suggests that a solution to the multiobjective identification problem exists. However, it is also shown that the affine local model structure is a highly sensitive parametrization when applied in transient operating regimes. Due to the multiobjective nature of the identification problem studied here, special considerations must be made during model structure selection, experiment design, and identification in order to meet both objectives. Some guidelines for experiment design are suggested and some robust nonlinear identification algorithms are studied. These include constrained and regularized identification and locally weighted identification. Their usefulness in the present context is illustrated by examples

Identification of nonlinear vibrating structures: Part I -- Formulation

A self-starting multistage, time-domain procedure is presented for the identification of nonlinear, multi-degree-of-freedom systems undergoing free oscillations or subjected to arbitrary direct force excitations and/or nonuniform support motions. Recursive least-squares parameter estimation methods combined with nonparametric identification techniques are used to represent, with sufficient accuracy, the identified system in a form that allows the convenient prediction of its transient response under excitations that differ from the test signals. The utility of this procedure is demonstrated in a companion paper

On errors-in-variables estimation with unknown noise variance ratio

We propose an estimation method for an errors-in-variables model with unknown input and output noise variances. The main assumption that allows identifiability of the model is clustering of the data into two clusters that are distinct in a certain specified sense. We show an application of the proposed method for system identification