1,126 research outputs found
Mathematical Basis for Physical Inference
While the axiomatic introduction of a probability distribution over a space
is common, its use for making predictions, using physical theories and prior
knowledge, suffers from a lack of formalization. We propose to introduce, in
the space of all probability distributions, two operations, the OR and the AND
operation, that bring to the space the necessary structure for making
inferences on possible values of physical parameters. While physical theories
are often asumed to be analytical, we argue that consistent inference needs to
replace analytical theories by probability distributions over the parameter
space, and we propose a systematic way of obtaining such "theoretical
correlations", using the OR operation on the results of physical experiments.
Predicting the outcome of an experiment or solving "inverse problems" are then
examples of the use of the AND operation. This leads to a simple and complete
mathematical basis for general physical inference.Comment: 24 pages, 4 figure
Informed Proposal Monte Carlo
Any search or sampling algorithm for solution of inverse problems needs
guidance to be efficient. Many algorithms collect and apply information about
the problem on the fly, and much improvement has been made in this way.
However, as a consequence of the the No-Free-Lunch Theorem, the only way we can
ensure a significantly better performance of search and sampling algorithms is
to build in as much information about the problem as possible. In the special
case of Markov Chain Monte Carlo sampling (MCMC) we review how this is done
through the choice of proposal distribution, and we show how this way of adding
more information about the problem can be made particularly efficient when
based on an approximate physics model of the problem. A highly nonlinear
inverse scattering problem with a high-dimensional model space serves as an
illustration of the gain of efficiency through this approach
Inconsistency and Acausality of Model Selection in Bayesian Inverse Problems
Bayesian inference paradigms are regarded as powerful tools for solution of
inverse problems. However, when applied to inverse problems in physical
sciences, Bayesian formulations suffer from a number of inconsistencies that
are often overlooked. A well known, but mostly neglected, difficulty is
connected to the notion of conditional probability densities. Borel, and later
Kolmogorov's (1933/1956), found that the traditional definition of conditional
densities is incomplete: In different parameterizations it leads to different
results. We will show an example where two apparently correct procedures
applied to the same problem lead to two widely different results. Another type
of inconsistency involves violation of causality. This problem is found in
model selection strategies in Bayesian inversion, such as Hierarchical Bayes
and Trans-Dimensional Inversion where so-called hyperparameters are included as
variables to control either the number (or type) of unknowns, or the prior
uncertainties on data or model parameters. For Hierarchical Bayes we
demonstrate that the calculated 'prior' distributions of data or model
parameters are not prior-, but posterior information. In fact, the calculated
'standard deviations' of the data are a measure of the inability of the forward
function to model the data, rather than uncertainties of the data. For
trans-dimensional inverse problems we show that the so-called evidence is, in
fact, not a measure of the success of fitting the data for the given choice (or
number) of parameters, as often claimed. We also find that the notion of
Natural Parsimony is ill-defined, because of its dependence on the parameter
prior. Based on this study, we find that careful rethinking of Bayesian
inversion practices is required, with special emphasis on ways of avoiding the
Borel-Kolmogorov inconsistency, and on the way we interpret model selection
results.Comment: The paper replaces arXiv:2308.05858v1 which contained incorrectly
normalized distributions in two important counterexamples on hierarchical
Bayes and trans-dimensional inversion. In the new, corrected version of the
paper (where a key counter-example on transdimensional inversion is further
expanded) the conclusions remain the same as in the original pape
- …