Demonstration of Enhanced Monte Carlo Computation of the Fisher Information for Complex Problems

The Fisher information matrix summarizes the amount of information in a set of data relative to the quantities of interest. There are many applications of the information matrix in statistical modeling, system identification and parameter estimation. This short paper reviews a feedback-based method and an independent perturbation approach for computing the information matrix for complex problems, where a closed form of the information matrix is not achievable. We show through numerical examples how these methods improve the accuracy of the estimate of the information matrix compared to the basic resampling-based approach. Some relevant theory is summarized

Information matrix for hidden Markov models with covariates

For a general class of hidden Markov models that may include time-varying covariates, we illustrate how to compute the observed information matrix, which may be used to obtain standard errors for the parameter estimates and check model identifiability. The proposed method is based on the Oakes’ identity and, as such, it allows for the exact computation of the information matrix on the basis of the output of the expectation-maximization (EM) algorithm for maximum likelihood estimation. In addition to this output, the method requires the first derivative of the posterior probabilities computed by the forward-backward recursions introduced by Baum and Welch. Alternative methods for computing exactly the observed information matrix require, instead, to differentiate twice the forward recursion used to compute the model likelihood, with a greater additional effort with respect to the EM algorithm. The proposed method is illustrated by a series of simulations and an application based on a longitudinal dataset in Health Economics

Fisher information matrix for single molecules with stochastic trajectories

Tracking of objects in cellular environments has become a vital tool in molecular cell biology. A particularly important example is single molecule tracking which enables the study of the motion of a molecule in cellular environments and provides quantitative information on the behavior of individual molecules in cellular environments, which were not available before through bulk studies. Here, we consider a dynamical system where the motion of an object is modeled by stochastic differential equations (SDEs), and measurements are the detected photons emitted by the moving fluorescently labeled object, which occur at discrete time points, corresponding to the arrival times of a Poisson process, in contrast to uniform time points which have been commonly used in similar dynamical systems. The measurements are distributed according to optical diffraction theory, and therefore, they would be modeled by different distributions, e.g., a Born and Wolf profile for an out-of-focus molecule. For some special circumstances, Gaussian image models have been proposed. In this paper, we introduce a stochastic framework in which we calculate the maximum likelihood estimates of the biophysical parameters of the molecular interactions, e.g., diffusion and drift coefficients. More importantly, we develop a general framework to calculate the Cram\'er-Rao lower bound (CRLB), given by the inverse of the Fisher information matrix, for the estimation of unknown parameters and use it as a benchmark in the evaluation of the standard deviation of the estimates. There exists no established method, even for Gaussian measurements, to systematically calculate the CRLB for the general motion model that we consider in this paper. We apply the developed methodology to simulated data of a molecule with linear trajectories and show that the standard deviation of the estimates matches well with the square root of the CRLB

Testing the Information Matrix Equality with Robust Estimators

We study the behaviour of the information matrix (IM) test when maximum likelihood estimators are replaced with robust estimators. The latter may unmask outliers and hence improve the power of the test. We investigate in detail the local asymptotic power of the IM test in the normal model, for various estimators and under a range of local alternatives. These local alternatives include contamination neighbourhoods, Student's t (with degrees of freedom approaching infinity), skewness, and a tilted normal. Simulation studies for fixed alternatives confirm that in many cases the use of robust estimators substantially increases the power of the IM test.