I Got More Data, My Model is More Refined, but My Estimator is Getting Worse! Am I Just Dumb?

Possibly, but more likely you are merely a victim of conventional wisdom. More data or better models by no means guarantee better estimators (e.g., with a smaller mean squared error), when you are not following probabilistically principled methods such as MLE (for large samples) or Bayesian approaches. Estimating equations are particularly vulnerable in this regard, almost a necessary price for their robustness. These points will be demonstrated via common tasks of estimating regression parameters and correlations, under simple models such as bivariate normal and ARCH(1). Some general strategies for detecting and avoiding such pitfalls are suggested, including checking for self-efficiency (Meng, 1994; Statistical Science) and adopting a guiding working model. Using the example of estimating the autocorrelation $\rho$ under a stationary AR(1) model, we also demonstrate the interaction between model assumptions and observation structures in seeking additional information, as the sampling interval $s$ increases. Furthermore, for a given sample size, the optimal s for minimizing the asymptotic variance of $\hat{\rho}_{MLE}$is $s = 1$ if and only if $\rho^2 ≤ 1/3$; beyond that region the optimal s increases at the rate of $log ^{−1}(\rho^{−2})$ as $\rho$ approaches a unit root, as does the gain in efficiency relative to using $s = 1$. A practical implication of this result is that the so-called “non-informative” Jeffreys prior can be far from non-informative even for stationary time series models, because here it converges rapidly to a point mass at a unit root as $s$ increases. Our overall emphasis is that intuition and conventional wisdom need to be examined via critical thinking and theoretical verification before they can be trusted fully.Statistic

Decomposition of structural learning about directed acyclic graphs

AbstractIn this paper, we propose that structural learning of a directed acyclic graph can be decomposed into problems related to its decomposed subgraphs. The decomposition of structural learning requires conditional independencies, but it does not require that separators are complete undirected subgraphs. Domain or prior knowledge of conditional independencies can be utilized to facilitate the decomposition of structural learning. By decomposition, search for d-separators in a large network is localized to small subnetworks. Thus both the efficiency of structural learning and the power of conditional independence tests can be improved

Structural Learning of Chain Graphs via Decomposition

Chain graphs present a broad class of graphical models for description of conditional independence structures, including both Markov networks and Bayesian networks as special cases. In this paper, we propose a computationally feasible method for the structural learning of chain graphs based on the idea of decomposing the learning problem into a set of smaller scale problems on its decomposed subgraphs. The decomposition requires conditional independencies but does not require the separators to be complete subgraphs. Algorithms for both skeleton recovery and complex arrow orientation are presented. Simulations under a variety of settings demonstrate the competitive performance of our method, especially when the underlying graph is sparse

Uniform Collapsibility of Distribution Dependence Over a Nominal, Ordinal or Continuous Background

Cox and Wermuth proposed that the partial derivative of the conditional distribution function of a random variable Y given another X is used for measuring association between two variables with arbitrary distributions. This paper shows the condition for collapsibility of the association measure

Optimal Shrinkage Estimation of Mean Parameters in Family of Distributions With Quadratic Variance

This paper discusses the simultaneous inference of mean parameters in a family of distributions with quadratic variance function. We first introduce a class of semiparametric/parametric shrinkage estimators and establish their asymptotic optimality properties. Two specific cases, the location-scale family and the natural exponential family with quadratic variance function, are then studied in detail. We conduct a comprehensive simulation study to compare the performance of the proposed methods with existing shrinkage estimators. We also apply the method to real data and obtain encouraging results

Ensemble Minimax Estimation for Multivariate Normal Means

This article discusses estimation of a heteroscedastic multivariate normal mean in terms of the ensemble risk. We first derive the ensemble minimaxity properties of various estimators that shrink towards zero. We then generalize our results to the case where the variances are given as a common unknown but estimable chi-squared random variable scaled by different known factors. We further provide a class of ensemble minimax estimators that shrink towards the common mea

Collapsibility for Directed Acyclic Graphs

Collapsibility means that the same statistical result of interest can be obtained before and after marginalization over some variables. In this paper, we discuss three kinds of collapsibility for directed acyclic graphs (DAGs): estimate collapsibility, conditional independence collapsibility and model collapsibility. Related to collapsibility, we discuss removability of variables from a DAG. We present conditions for these three different kinds of collapsibility and relationships among them. We give algorithms to find a minimum variable set containing a variable subset of interest onto which a statistical result is collapsible. Copyright (c) 2009 Board of the Foundation of the Scandinavian Journal of Statistics.