2,462 research outputs found
A variational Bayesian method for inverse problems with impulsive noise
We propose a novel numerical method for solving inverse problems subject to
impulsive noises which possibly contain a large number of outliers. The
approach is of Bayesian type, and it exploits a heavy-tailed t distribution for
data noise to achieve robustness with respect to outliers. A hierarchical model
with all hyper-parameters automatically determined from the given data is
described. An algorithm of variational type by minimizing the Kullback-Leibler
divergence between the true posteriori distribution and a separable
approximation is developed. The numerical method is illustrated on several one-
and two-dimensional linear and nonlinear inverse problems arising from heat
conduction, including estimating boundary temperature, heat flux and heat
transfer coefficient. The results show its robustness to outliers and the fast
and steady convergence of the algorithm.Comment: 20 pages, to appear in J. Comput. Phy
Joint state-parameter estimation of a nonlinear stochastic energy balance model from sparse noisy data
While nonlinear stochastic partial differential equations arise naturally in
spatiotemporal modeling, inference for such systems often faces two major
challenges: sparse noisy data and ill-posedness of the inverse problem of
parameter estimation. To overcome the challenges, we introduce a strongly
regularized posterior by normalizing the likelihood and by imposing physical
constraints through priors of the parameters and states. We investigate joint
parameter-state estimation by the regularized posterior in a physically
motivated nonlinear stochastic energy balance model (SEBM) for paleoclimate
reconstruction. The high-dimensional posterior is sampled by a particle Gibbs
sampler that combines MCMC with an optimal particle filter exploiting the
structure of the SEBM. In tests using either Gaussian or uniform priors based
on the physical range of parameters, the regularized posteriors overcome the
ill-posedness and lead to samples within physical ranges, quantifying the
uncertainty in estimation. Due to the ill-posedness and the regularization, the
posterior of parameters presents a relatively large uncertainty, and
consequently, the maximum of the posterior, which is the minimizer in a
variational approach, can have a large variation. In contrast, the posterior of
states generally concentrates near the truth, substantially filtering out
observation noise and reducing uncertainty in the unconstrained SEBM
Connections Between Adaptive Control and Optimization in Machine Learning
This paper demonstrates many immediate connections between adaptive control
and optimization methods commonly employed in machine learning. Starting from
common output error formulations, similarities in update law modifications are
examined. Concepts in stability, performance, and learning, common to both
fields are then discussed. Building on the similarities in update laws and
common concepts, new intersections and opportunities for improved algorithm
analysis are provided. In particular, a specific problem related to higher
order learning is solved through insights obtained from these intersections.Comment: 18 page
Centralized calibration of power system dynamic models using variational data assimilation
This paper presents a novel centralized, variational data assimilation
approach for calibrating transient dynamic models in electrical power systems,
focusing on load model parameters. With the increasing importance of
inverter-based resources, assessing power systems' dynamic performance under
disturbances has become challenging, necessitating robust model calibration
methods. The proposed approach expands on previous Bayesian frameworks by
establishing a posterior distribution of parameters using an approximation
around the maximum a posteriori value. We illustrate the efficacy of our method
by generating events of varying intensity, highlighting its ability to capture
the systems' evolution accurately and with associated uncertainty estimates.
This research improves the precision of dynamic performance assessments in
modern power systems, with potential applications in managing uncertainties and
optimizing system operations.Comment: 9 pages, 8 figures, and 1 tabl
Inferring unknown unknowns: Regularized bias-aware ensemble Kalman filter
Because of physical assumptions and numerical approximations, low-order
models are affected by uncertainties in the state and parameters, and by model
biases. Model biases, also known as model errors or systematic errors, are
difficult to infer because they are `unknown unknowns', i.e., we do not
necessarily know their functional form a priori. With biased models, data
assimilation methods may be ill-posed because either (i) they are
'bias-unaware' because the estimators are assumed unbiased, (ii) they rely on
an a priori parametric model for the bias, or (iii) they can infer model biases
that are not unique for the same model and data. First, we design a data
assimilation framework to perform combined state, parameter, and bias
estimation. Second, we propose a mathematical solution with a sequential
method, i.e., the regularized bias-aware ensemble Kalman Filter (r-EnKF), which
requires a model of the bias and its gradient (i.e., the Jacobian). Third, we
propose an echo state network as the model bias estimator. We derive the
Jacobian of the network, and design a robust training strategy with data
augmentation to accurately infer the bias in different scenarios. Fourth, we
apply the r-EnKF to nonlinearly coupled oscillators (with and without
time-delay) affected by different forms of bias. The r-EnKF infers in real-time
parameters and states, and a unique bias. The applications that we showcase are
relevant to acoustics, thermoacoustics, and vibrations; however, the r-EnKF
opens new opportunities for combined state, parameter and bias estimation for
real-time and on-the-fly prediction in nonlinear systems.Comment: 22 Figure
Approximate Bayesian Smoothing with Unknown Process and Measurement Noise Covariances
We present an adaptive smoother for linear state-space models with unknown
process and measurement noise covariances. The proposed method utilizes the
variational Bayes technique to perform approximate inference. The resulting
smoother is computationally efficient, easy to implement, and can be applied to
high dimensional linear systems. The performance of the algorithm is
illustrated on a target tracking example.Comment: Derivations for the smoother can found here:
http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-12070
- …