Cosmic Statistics of Statistics

The errors on statistics measured in finite galaxy catalogs are exhaustively investigated. The theory of errors on factorial moments by Szapudi & Colombi (1996) is applied to cumulants via a series expansion method. All results are subsequently extended to the weakly non-linear regime. Together with previous investigations this yields an analytic theory of the errors for moments and connected moments of counts in cells from highly nonlinear to weakly nonlinear scales. The final analytic formulae representing the full theory are explicit but somewhat complicated. Therefore as a companion to this paper we supply a FORTRAN program capable of calculating the described quantities numerically (abridged).Comment: 18 pages, 9 figures, Latex (MN format), published in MNRAS 310, 428 with slight correction

Shrinkage Estimation of the Power Spectrum Covariance Matrix

We seek to improve estimates of the power spectrum covariance matrix from a limited number of simulations by employing a novel statistical technique known as shrinkage estimation. The shrinkage technique optimally combines an empirical estimate of the covariance with a model (the target) to minimize the total mean squared error compared to the true underlying covariance. We test this technique on N-body simulations and evaluate its performance by estimating cosmological parameters. Using a simple diagonal target, we show that the shrinkage estimator significantly outperforms both the empirical covariance and the target individually when using a small number of simulations. We find that reducing noise in the covariance estimate is essential for properly estimating the values of cosmological parameters as well as their confidence intervals. We extend our method to the jackknife covariance estimator and again find significant improvement, though simulations give better results. Even for thousands of simulations we still find evidence that our method improves estimation of the covariance matrix. Because our method is simple, requires negligible additional numerical effort, and produces superior results, we always advocate shrinkage estimation for the covariance of the power spectrum and other large-scale structure measurements when purely theoretical modeling of the covariance is insufficient.Comment: 9 pages, 7 figures (1 new), MNRAS, accepted. Changes to match accepted version, including an additional explanatory section with 1 figur

The Angular Power Spectrum of the First-Year WMAP Data Reanalysed

We measure the angular power spectrum of the WMAP first-year temperature anisotropy maps. We use SpICE (Spatially Inhomogeneous Correlation Estimator) to estimate Cl's for multipoles l=2-900 from all possible cross-correlation channels. Except for the map-making stage, our measurements provide an independent analysis of that by Hinshaw etal (2003). Despite the different methods used, there is virtually no difference between the two measurements for l < 700 ; the highest l's are still compatible within 1-sigma errors. We use a novel intra-bin variance method to constrain Cl errors in a model independent way. When applied to WMAP data, the intra-bin variance estimator yields diagonal errors 10% larger than those reported by the WMAP team for 100 < l < 450. This translates into a 2.4 sigma detection of systematics since no difference is expected between the SpICE and the WMAP team estimator window functions in this multipole range. With our measurement of the Cl's and errors, we get chi^2/d.o.f. = 1.042 for a best-fit LCDM model, which has a 14% probability, whereas the WMAP team obtained chi^2/d.o.f. = 1.066, which has a 5% probability. We assess the impact of our results on cosmological parameters using Markov Chain Monte Carlo simulations. From WMAP data alone, assuming spatially flat power law LCDM models, we obtain the reionization optical depth tau = 0.145 +/- 0.067, spectral index n_s = 0.99 +/- 0.04, Hubble constant h = 0.67 +/- 0.05, baryon density Omega_b h^2 = 0.0218 +/- 0.0014, cold dark matter density Omega_{cdm} h^2 = 0.122 +/- 0.018, and sigma_8 = 0.92 +/- 0.12, consistent with a reionization redshift z_{re} = 16 +/- 5 (68% CL).Comment: Matches version accepted by ApJ Letters. Main changes: emphasizes chi2 value for best-fit model given our estimate of Cls and errors vs. WMAP team's. Potential detection of systematics in WMAP data quantified. Power spectrum and other data files available at http://www.ifa.hawaii.edu/cosmowave/wmap.htm

Comparison of the Large Scale Clustering in the APM and the EDSGC Galaxy Surveys

Clustering statistics are compared in the Automatic Plate Machine (APM) and the Edinburgh/Durham Southern Galaxy Catalogue (EDSGC) angular galaxy surveys. Both surveys were independently constructed from scans of the same adjacent UK IIIa--J Schmidt photographic plates with the APM and COSMOS microdensitometers, respectively. The comparison of these catalogs is a rare practical opportunity to study systematic errors, which cannot be achieved via simulations or theoretical methods. On intermediate scales, $0.1^\circ < \theta < 0.5^\circ$, we find good agreement for the cumulants or reduced moments of counts in cells up to sixth order. On larger scales there is a small disagreement due to edge effects in the EDSGC, which covers a smaller area. On smaller scales, we find a significant disagreement that can only be attributed to differences in the construction of the surveys, most likely the dissimilar deblending of crowded fields. The overall agreement of the APM and EDSGC is encouraging, and shows that the results for intermediate scales should be fairly robust. On the other hand, the systematic deviations found at small scales are significant in a regime, where comparison with theory and simulations is possible. This is an important fact to bear in mind when planning the construction of future digitized galaxy catalogs.Comment: 4 pages with 3 figures included. Submitted for MNRAS 'pink pages

Star-galaxy separation strategies for WISE-2MASS all-sky infrared galaxy catalogs

We combine photometric information of the WISE and 2MASS all-sky infrared databases, and demonstrate how to produce clean and complete galaxy catalogs for future analyses. Adding 2MASS colors to WISE photometry improves star-galaxy separation efficiency substantially at the expense of loosing a small fraction of the galaxies. We find that 93% of the WISE objects within W1<15.2 mag have a 2MASS match, and that a class of supervised machine learning algorithms, Support Vector Machines (SVM), are efficient classifiers of objects in our multicolor data set. We constructed a training set from the SDSS PhotoObj table with known star-galaxy separation, and determined redshift distribution of our sample from the GAMA spectroscopic survey. Varying the combination of photometric parameters input into our algorithm we show that W1 - J is a simple and effective star-galaxy separator, capable of producing results comparable to the multi-dimensional SVM classification. We present a detailed description of our star-galaxy separation methods, and characterize the robustness of our tools in terms of contamination, completeness, and accuracy. We explore systematics of the full sky WISE-2MASS galaxy map, such as contamination from Moon glow. We show that the homogeneity of the full sky galaxy map is improved by an additional J<16.5 mag flux limit. The all-sky galaxy catalog we present in this paper covers 21,200 sq. degrees with dusty regions masked out, and has an estimated stellar contamination of 1.2% and completeness of 70.1% among 2.4 million galaxies with $z_{med}= 0.14$. WISE-2MASS galaxy maps with well controlled stellar contamination will be useful for spatial statistical analyses, including cross correlations with other cosmological random fields, such as the Cosmic Microwave Background. The same techniques also yield a statistically controlled sample of stars as well.Comment: 10 pages, 11 figures. Accepted for publication in MNRA