Biased-estimations of the Variance and Skewness

Nonlinear combinations of direct observables are often used to estimate quantities of theoretical interest. Without sufficient caution, this could lead to biased estimations. An example of great interest is the skewness $S_3$ of the galaxy distribution, defined as the ratio of the third moment \xibar_3 and the variance squared \xibar_2^2. Suppose one is given unbiased estimators for \xibar_3 and \xibar_2^2 respectively, taking a ratio of the two does not necessarily result in an unbiased estimator of $S_3$. Exactly such an estimation-bias affects most existing measurements of $S_3$. Furthermore, common estimators for \xibar_3 and \xibar_2 suffer also from this kind of estimation-bias themselves: for \xibar_2, it is equivalent to what is commonly known as the integral constraint. We present a unifying treatment allowing all these estimation-biases to be calculated analytically. They are in general negative, and decrease in significance as the survey volume increases, for a given smoothing scale. We present a re-analysis of some existing measurements of the variance and skewness and show that most of the well-known systematic discrepancies between surveys with similar selection criteria, but different sizes, can be attributed to the volume-dependent estimation-biases. This affects the inference of the galaxy-bias(es) from these surveys. Our methodology can be adapted to measurements of analogous quantities in quasar spectra and weak-lensing maps. We suggest methods to reduce the above estimation-biases, and point out other examples in LSS studies which might suffer from the same type of a nonlinear-estimation-bias.Comment: 28 pages of text, 9 ps figures, submitted to Ap

Systematic inference of the long-range dependence and heavy-tail distribution parameters of ARFIMA models

Long-Range Dependence (LRD) and heavy-tailed distributions are ubiquitous in natural and socio-economic data. Such data can be self-similar whereby both LRD and heavy-tailed distributions contribute to the self-similarity as measured by the Hurst exponent. Some methods widely used in the physical sciences separately estimate these two parameters, which can lead to estimation bias. Those which do simultaneous estimation are based on frequentist methods such as Whittle’s approximate maximum likelihood estimator. Here we present a new and systematic Bayesian framework for the simultaneous inference of the LRD and heavy-tailed distribution parameters of a parametric ARFIMA model with non-Gaussian innovations. As innovations we use the α-stable and t-distributions which have power law tails. Our algorithm also provides parameter uncertainty estimates. We test our algorithm using synthetic data, and also data from the Geostationary Operational Environmental Satellite system (GOES) solar X-ray time series. These tests show that our algorithm is able to accurately and robustly estimate the LRD and heavy-tailed distribution parameters

The effect of round-off error on long memory processes

We study how the round-off (or discretization) error changes the statistical properties of a Gaussian long memory process. We show that the autocovariance and the spectral density of the discretized process are asymptotically rescaled by a factor smaller than one, and we compute exactly this scaling factor. Consequently, we find that the discretized process is also long memory with the same Hurst exponent as the original process. We consider the properties of two estimators of the Hurst exponent, namely the local Whittle (LW) estimator and the Detrended Fluctuation Analysis (DFA). By using analytical considerations and numerical simulations we show that, in presence of round-off error, both estimators are severely negatively biased in finite samples. Under regularity conditions we prove that the LW estimator applied to discretized processes is consistent and asymptotically normal. Moreover, we compute the asymptotic properties of the DFA for a generic (i.e. non Gaussian) long memory process and we apply the result to discretized processes.Comment: 44 pages, 4 figures, 4 table

Econometrics: A Bird’s Eye View

As a unified discipline, econometrics is still relatively young and has been transforming and expanding very rapidly over the past few decades. Major advances have taken place in the analysis of cross sectional data by means of semi-parametric and non-parametric techniques. Heterogeneity of economic relations across individuals, firms and industries is increasingly acknowledged and attempts have been made to take them into account either by integrating out their effects or by modeling the sources of heterogeneity when suitable panel data exists. The counterfactual considerations that underlie policy analysis and treatment evaluation have been given a more satisfactory foundation. New time series econometric techniques have been developed and employed extensively in the areas of macroeconometrics and finance. Non-linear econometric techniques are used increasingly in the analysis of cross section and time series observations. Applications of Bayesian techniques to econometric problems have been given new impetus largely thanks to advances in computer power and computational techniques. The use of Bayesian techniques have in turn provided the investigators with a unifying framework where the tasks of forecasting, decision making, model evaluation and learning can be considered as parts of the same interactive and iterative process; thus paving the way for establishing the foundation of “real time econometrics”. This paper attempts to provide an overview of some of these developments.history of econometrics, microeconometrics, macroeconometrics, Bayesian econometrics, nonparametric and semi-parametric analysis

Tests of Statistical Methods for Estimating Galaxy Luminosity Function and Applications to the Hubble Deep Field

We studied the statistical methods for the estimation of the luminosity function (LF) of galaxies. We focused on four nonparametric estimators: $1/V_{\rm max}$ estimator, maximum-likelihood estimator of Efstathiou et al. (1988), Cho{\l}oniewski's estimator, and improved Lynden-Bell's estimator. The performance of the $1/V_{\rm max}$ estimator has been recently questioned, especially for the faint-end estimation of the LF. We improved these estimators for the studies of the distant Universe, and examined their performances for various classes of functional forms by Monte Carlo simulations. We also applied these estimation methods to the mock 2dF redshift survey catalog prepared by Cole et al. (1998). We found that $1/V_{\rm max}$ estimator yields a completely unbiased result if there is no inhomogeneity, but is not robust against clusters or voids. This is consistent with the well-known results, and we did not confirm the bias trend of $1/V_{\rm max}$ estimator claimed by Willmer (1997) in the case of homogeneous sample. We also found that the other three maximum-likelihood type estimators are quite robust and give consistent results with each other. In practice we recommend Cho{\l}oniewski's estimator for two reasons: 1. it simultaneously provides the shape and normalization of the LF; 2. it is the fastest among these four estimators, because of the algorithmic simplicity. Then, we analyzed the photometric redshift data of the Hubble Deep Field prepared by Fern\'{a}ndez-Soto et al. (1999) using the above four methods. We also derived luminosity density $\rho_{\rm L}$ at $B$- and $I$-band. Our $B$-band estimation is roughly consistent with that of Sawicki, Lin, & Yee (1997), but a few times lower at $2.0 < z < 3.0$. The evolution of $\rho_{\rm L}(I)$ is found to be less prominent.Comment: To appear in ApJS July 2000 issue. 36 page