20,822 research outputs found
Approximate maximum likelihood estimation using data-cloning ABC
A maximum likelihood methodology for a general class of models is presented,
using an approximate Bayesian computation (ABC) approach. The typical target of
ABC methods are models with intractable likelihoods, and we combine an ABC-MCMC
sampler with so-called "data cloning" for maximum likelihood estimation.
Accuracy of ABC methods relies on the use of a small threshold value for
comparing simulations from the model and observed data. The proposed
methodology shows how to use large threshold values, while the number of
data-clones is increased to ease convergence towards an approximate maximum
likelihood estimate. We show how to exploit the methodology to reduce the
number of iterations of a standard ABC-MCMC algorithm and therefore reduce the
computational effort, while obtaining reasonable point estimates. Simulation
studies show the good performance of our approach on models with intractable
likelihoods such as g-and-k distributions, stochastic differential equations
and state-space models.Comment: 25 pages. Minor revision. It includes a parametric bootstrap for the
exact MLE for the first example; includes mean bias and RMSE calculations for
the third example. Forthcoming in Computational Statistics and Data Analysi
Fixed effects selection in the linear mixed-effects model using adaptive ridge procedure for L0 penalty performance
This paper is concerned with the selection of fixed effects along with the
estimation of fixed effects, random effects and variance components in the
linear mixed-effects model. We introduce a selection procedure based on an
adaptive ridge (AR) penalty of the profiled likelihood, where the covariance
matrix of the random effects is Cholesky factorized. This selection procedure
is intended to both low and high-dimensional settings where the number of fixed
effects is allowed to grow exponentially with the total sample size, yielding
technical difficulties due to the non-convex optimization problem induced by L0
penalties. Through extensive simulation studies, the procedure is compared to
the LASSO selection and appears to enjoy the model selection consistency as
well as the estimation consistency
Expectation Propagation for Nonlinear Inverse Problems -- with an Application to Electrical Impedance Tomography
In this paper, we study a fast approximate inference method based on
expectation propagation for exploring the posterior probability distribution
arising from the Bayesian formulation of nonlinear inverse problems. It is
capable of efficiently delivering reliable estimates of the posterior mean and
covariance, thereby providing an inverse solution together with quantified
uncertainties. Some theoretical properties of the iterative algorithm are
discussed, and the efficient implementation for an important class of problems
of projection type is described. The method is illustrated with one typical
nonlinear inverse problem, electrical impedance tomography with complete
electrode model, under sparsity constraints. Numerical results for real
experimental data are presented, and compared with that by Markov chain Monte
Carlo. The results indicate that the method is accurate and computationally
very efficient.Comment: Journal of Computational Physics, to appea
A variational Bayesian method for inverse problems with impulsive noise
We propose a novel numerical method for solving inverse problems subject to
impulsive noises which possibly contain a large number of outliers. The
approach is of Bayesian type, and it exploits a heavy-tailed t distribution for
data noise to achieve robustness with respect to outliers. A hierarchical model
with all hyper-parameters automatically determined from the given data is
described. An algorithm of variational type by minimizing the Kullback-Leibler
divergence between the true posteriori distribution and a separable
approximation is developed. The numerical method is illustrated on several one-
and two-dimensional linear and nonlinear inverse problems arising from heat
conduction, including estimating boundary temperature, heat flux and heat
transfer coefficient. The results show its robustness to outliers and the fast
and steady convergence of the algorithm.Comment: 20 pages, to appear in J. Comput. Phy
Recommended from our members
Reply to comment by K. Beven and P. Young on "Bayesian recursive parameter estimation for hydrologic models''
Getting Started with Particle Metropolis-Hastings for Inference in Nonlinear Dynamical Models
This tutorial provides a gentle introduction to the particle
Metropolis-Hastings (PMH) algorithm for parameter inference in nonlinear
state-space models together with a software implementation in the statistical
programming language R. We employ a step-by-step approach to develop an
implementation of the PMH algorithm (and the particle filter within) together
with the reader. This final implementation is also available as the package
pmhtutorial in the CRAN repository. Throughout the tutorial, we provide some
intuition as to how the algorithm operates and discuss some solutions to
problems that might occur in practice. To illustrate the use of PMH, we
consider parameter inference in a linear Gaussian state-space model with
synthetic data and a nonlinear stochastic volatility model with real-world
data.Comment: 41 pages, 7 figures. In press for Journal of Statistical Software.
Source code for R, Python and MATLAB available at:
https://github.com/compops/pmh-tutoria
Estimation of a regression spline sample selection model
It is often the case that an outcome of interest is observed for a restricted non-randomly selected sample of the population. In such a situation, standard statistical analysis yields biased results. This issue can be addressed using sample selection models which are based on the estimation of two regressions: a binary selection equation determining whether a particular statistical unit will be available in the outcome equation. Classic sample selection models assume a priori that continuous regressors have a pre-specified linear or non-linear relationship to the outcome, which can lead to erroneous conclusions. In the case of continuous response, methods in which covariate effects are modeled flexibly have been previously proposed, the most recent being based on a Bayesian Markov chain Monte Carlo approach. A frequentist counterpart which has the advantage of being computationally fast is introduced. The proposed algorithm is based on the penalized likelihood estimation framework. The construction of confidence intervals is also discussed. The empirical properties of the existing and proposed methods are studied through a simulation study. The approaches are finally illustrated by analyzing data from the RAND Health Insurance Experiment on annual health expenditures
- …