53,560 research outputs found
Graphical continuous Lyapunov models
The linear Lyapunov equation of a covariance matrix parametrizes the
equilibrium covariance matrix of a stochastic process. This parametrization can
be interpreted as a new graphical model class, and we show how the model class
behaves under marginalization and introduce a method for structure learning via
-penalized loss minimization. Our proposed method is demonstrated to
outperform alternative structure learning algorithms in a simulation study, and
we illustrate its application for protein phosphorylation network
reconstruction.Comment: 10 pages, 5 figure
Probabilistic Interpretation of Linear Solvers
This manuscript proposes a probabilistic framework for algorithms that
iteratively solve unconstrained linear problems with positive definite
for . The goal is to replace the point estimates returned by existing
methods with a Gaussian posterior belief over the elements of the inverse of
, which can be used to estimate errors. Recent probabilistic interpretations
of the secant family of quasi-Newton optimization algorithms are extended.
Combined with properties of the conjugate gradient algorithm, this leads to
uncertainty-calibrated methods with very limited cost overhead over conjugate
gradients, a self-contained novel interpretation of the quasi-Newton and
conjugate gradient algorithms, and a foundation for new nonlinear optimization
methods.Comment: final version, in press at SIAM J Optimizatio
Generalized Stochastic Gradient Learning
We study the properties of generalized stochastic gradient (GSG) learning in forwardlooking models. We examine how the conditions for stability of standard stochastic gradient (SG) learning both di1er from and are related to E-stability, which governs stability under least squares learning. SG algorithms are sensitive to units of measurement and we show that there is a transformation of variables for which E-stability governs SG stability. GSG algorithms with constant gain have a deeper justification in terms of parameter drift, robustness and risk sensitivity
Distributional Robustness of K-class Estimators and the PULSE
Recently, in causal discovery, invariance properties such as the moment
criterion which two-stage least square estimator leverage have been exploited
for causal structure learning: e.g., in cases, where the causal parameter is
not identifiable, some structure of the non-zero components may be identified,
and coverage guarantees are available. Subsequently, anchor regression has been
proposed to trade-off invariance and predictability. The resulting estimator is
shown to have optimal predictive performance under bounded shift interventions.
In this paper, we show that the concepts of anchor regression and K-class
estimators are closely related. Establishing this connection comes with two
benefits: (1) It enables us to prove robustness properties for existing K-class
estimators when considering distributional shifts. And, (2), we propose a novel
estimator in instrumental variable settings by minimizing the mean squared
prediction error subject to the constraint that the estimator lies in an
asymptotically valid confidence region of the causal parameter. We call this
estimator PULSE (p-uncorrelated least squares estimator) and show that it can
be computed efficiently, even though the underlying optimization problem is
non-convex. We further prove that it is consistent. We perform simulation
experiments illustrating that there are several settings including weak
instrument settings, where PULSE outperforms other estimators and suffers from
less variability.Comment: 85 pages, 15 figure
- …