Limit theorems for some adaptive MCMC algorithms with subgeometric kernels: Part II

We prove a central limit theorem for a general class of adaptive Markov Chain Monte Carlo algorithms driven by sub-geometrically ergodic Markov kernels. We discuss in detail the special case of stochastic approximation. We use the result to analyze the asymptotic behavior of an adaptive version of the Metropolis Adjusted Langevin algorithm with a heavy tailed target density.Comment: 34 page

Estimation of Network structures from partially observed Markov random fields

We consider the estimation of high-dimensional network structures from partially observed Markov random field data using a penalized pseudo-likelihood approach. We fit a misspecified model obtained by ignoring the missing data problem. We study the consistency of the estimator and derive a bound on its rate of convergence. The results obtained relate the rate of convergence of the estimator to the extent of the missing data problem. We report some simulation results that empirically validate some of the theoretical findings.Comment: 24 pages 1 figur

EFFICIENCY BOUNDS FOR SEMIPARAMETRIC MODELS WITH SINGULAR SCORE FUNCTIONS

This paper is concerned with asymptotic efficiency bounds for the estimation of the finite dimension parameter \theta\in \rset^p of semiparametric models that have singular score function for $\theta$ at the true value $\theta_\star$. The resulting singularity of the matrix of Fisher information means that the standard bound derived by Begun et. al. ([1]) for $\theta-\theta_\star$ is not defined. We study the case of single rank deficiency of the score and focus on the case where the derivative of the root density in the direction of the last parameter component, $\theta_2$, is nil while the derivatives in the $p-1$ other directions, $\theta_1$, are linearly independent. We then distinguish two cases: (i) The second derivative of the root density in the direction of $\theta_2$ and the first derivative in the direction of $\theta_1$ are linearly independent and (ii) The second derivative of the root density in the direction of $\theta_2$ is also nil but the third derivative in $\theta_2$ is linearly independent of the first derivative in the direction of $\theta_1$. We show that in both cases, efficiency bounds can be obtained for the estimation of $\kappa_j(\theta)=(\theta_1-\theta_{\star1}, (\theta_2-\theta_{\star2})^j)$, with $j=2$ and 3, respectively and argue that an estimator $\hat{\theta}$ is efficient if $\kappa_j(\hat{\theta})$ reaches its bound. We provide the bounds in form of convolution and asymptotic minimax theorems. For case (i), we propose a transformation of the Gaussian variable that appears in our convolution theorem to account for the restricted set of values of $\kappa_2(\theta)$. This transformation effectively gives the efficiency bound for the estimation of $\kappa_2(\theta)$ in the model configuration (i). We apply these results to locally under-identified moment condition models and show that the generalized method of moment (GMM) estimator using $V_\star^{-1}$ as weighting matrix, where $V_\star$ is the variance of the estimating function, is optimal even in these non standard settings

Estimation of high-dimensional partially-observed discrete Markov random fields

Abstract. We consider the problem of estimating the parameters of discrete Markov random fields from partially observed data in a high-dimensional setting. Using a `1-penalized pseudo-likelihood approach, we fit a misspecified model obtained by ignoring the missing data problem. We derive an estimation error bound that highlights the effect of the misspecification. We report some simulation results that illustrate the theoretical findings