14,107 research outputs found
Bayesian inference for inverse problems
Traditionally, the MaxEnt workshops start by a tutorial day. This paper
summarizes my talk during 2001'th workshop at John Hopkins University. The main
idea in this talk is to show how the Bayesian inference can naturally give us
all the necessary tools we need to solve real inverse problems: starting by
simple inversion where we assume to know exactly the forward model and all the
input model parameters up to more realistic advanced problems of myopic or
blind inversion where we may be uncertain about the forward model and we may
have noisy data. Starting by an introduction to inverse problems through a few
examples and explaining their ill posedness nature, I briefly presented the
main classical deterministic methods such as data matching and classical
regularization methods to show their limitations. I then presented the main
classical probabilistic methods based on likelihood, information theory and
maximum entropy and the Bayesian inference framework for such problems. I show
that the Bayesian framework, not only generalizes all these methods, but also
gives us natural tools, for example, for inferring the uncertainty of the
computed solutions, for the estimation of the hyperparameters or for handling
myopic or blind inversion problems. Finally, through a deconvolution problem
example, I presented a few state of the art methods based on Bayesian inference
particularly designed for some of the mass spectrometry data processing
problems.Comment: Presented at MaxEnt01. To appear in Bayesian Inference and Maximum
Entropy Methods, B. Fry (Ed.), AIP Proceedings. 20pages, 13 Postscript
figure
Takeuchi's Information Criteria as a form of Regularization
Takeuchi's Information Criteria (TIC) is a linearization of maximum
likelihood estimator bias which shrinks the model parameters towards the
maximum entropy distribution, even when the model is mis-specified. In
statistical machine learning, regularization (a.k.a. ridge regression)
also introduces a parameterized bias term with the goal of minimizing
out-of-sample entropy, but generally requires a numerical solver to find the
regularization parameter. This paper presents a novel regularization approach
based on TIC; the approach does not assume a data generation process and
results in a higher entropy distribution through more efficient sample noise
suppression. The resulting objective function can be directly minimized to
estimate and select the best model, without the need to select a regularization
parameter, as in ridge regression. Numerical results applied to a synthetic
high dimensional dataset generated from a logistic regression model demonstrate
superior model performance when using the TIC based regularization over a
and a penalty term
Traffic matrix estimation on a large IP backbone: a comparison on real data
This paper considers the problem of estimating the point-to-point
traffic matrix in an operational IP backbone. Contrary to previous studies, that have used a partial traffic matrix or demands estimated from aggregated Netflow traces, we use a unique data set of complete traffic matrices from a global IP network measured over five-minute intervals. This allows us to do an accurate data analysis on the time-scale of typical link-load measurements and enables us to make a balanced evaluation of different traffic matrix estimation techniques. We describe the data collection infrastructure, present spatial and temporal demand distributions, investigate the stability of fan-out factors, and analyze the mean-variance relationships between demands. We perform a critical evaluation of existing and novel methods for traffic matrix estimation, including recursive fanout estimation, worst-case bounds, regularized estimation techniques, and methods that rely on mean-variance relationships. We discuss the weaknesses and strengths of the various methods, and highlight differences in the results for the European and American subnetworks
- …