RascalC: A Jackknife Approach to Estimating Single and Multi-Tracer Galaxy Covariance Matrices

To make use of clustering statistics from large cosmological surveys, accurate and precise covariance matrices are needed. We present a new code to estimate large scale galaxy two-point correlation function (2PCF) covariances in arbitrary survey geometries that, due to new sampling techniques, runs $\sim 10^4$ times faster than previous codes, computing finely-binned covariance matrices with negligible noise in less than 100 CPU-hours. As in previous works, non-Gaussianity is approximated via a small rescaling of shot-noise in the theoretical model, calibrated by comparing jackknife survey covariances to an associated jackknife model. The flexible code, RascalC, has been publicly released, and automatically takes care of all necessary pre- and post-processing, requiring only a single input dataset (without a prior 2PCF model). Deviations between large scale model covariances from a mock survey and those from a large suite of mocks are found to be be indistinguishable from noise. In addition, the choice of input mock are shown to be irrelevant for desired noise levels below $\sim 10^5$ mocks. Coupled with its generalization to multi-tracer data-sets, this shows the algorithm to be an excellent tool for analysis, reducing the need for large numbers of mock simulations to be computed.Comment: 29 pages, 8 figures. Accepted by MNRAS. Code is available at http://github.com/oliverphilcox/RascalC with documentation at http://rascalc.readthedocs.io

A conceptual proposal on the undecidability of the distribution law of prime numbers and theoretical consequences

Within the conceptual framework of number theory, we consider prime numbers and the classic still unsolved problem to find a complete law of their distribution. We ask ourselves if such persisting difficulties could be understood as due to theoretical incompatibilities. We consider the problem in the conceptual framework of computational theory. This article is a contribution to the philosophy of mathematics proposing different possible understandings of the supposed theoretical unavailability and indemonstrability of the existence of a law of distribution of prime numbers. Tentatively, we conceptually consider demonstrability as computability, in our case the conceptual availability of an algorithm able to compute the general properties of the presumed primes’ distribution law without computing such distribution. The link between the conceptual availability of a distribution law of primes and decidability is given by considering how to decide if a number is prime without computing. The supposed distribution law should allow for any given prime knowing the next prime without factorial computing. Factorial properties of numbers, such as their property of primality, require their factorisation (or equivalent, e.g., the sieves), i.e., effective computing. However, we have factorisation techniques available, but there are no (non-quantum) known algorithms which can effectively factor arbitrary large integers. Then factorisation is undecidable. We consider the theoretical unavailability of a distribution law for factorial properties, as being prime, equivalent to its non-computability, undecidability. The availability and demonstrability of a hypothetical law of distribution of primes is inconsistent with its undecidability. The perspective is to transform this conjecture into a theorem

Computing hypergeometric functions rigorously

We present an efficient implementation of hypergeometric functions in arbitrary-precision interval arithmetic. The functions ${}_0F_1$, ${}_1F_1$, ${}_2F_1$ and ${}_2F_0$ (or the Kummer $U$-function) are supported for unrestricted complex parameters and argument, and by extension, we cover exponential and trigonometric integrals, error functions, Fresnel integrals, incomplete gamma and beta functions, Bessel functions, Airy functions, Legendre functions, Jacobi polynomials, complete elliptic integrals, and other special functions. The output can be used directly for interval computations or to generate provably correct floating-point approximations in any format. Performance is competitive with earlier arbitrary-precision software, and sometimes orders of magnitude faster. We also partially cover the generalized hypergeometric function ${}_pF_q$ and computation of high-order parameter derivatives.Comment: v2: corrected example in section 3.1; corrected timing data for case E-G in section 8.5 (table 6, figure 2); adjusted paper siz

Parallel Algorithms for Summing Floating-Point Numbers

The problem of exactly summing n floating-point numbers is a fundamental problem that has many applications in large-scale simulations and computational geometry. Unfortunately, due to the round-off error in standard floating-point operations, this problem becomes very challenging. Moreover, all existing solutions rely on sequential algorithms which cannot scale to the huge datasets that need to be processed. In this paper, we provide several efficient parallel algorithms for summing n floating point numbers, so as to produce a faithfully rounded floating-point representation of the sum. We present algorithms in PRAM, external-memory, and MapReduce models, and we also provide an experimental analysis of our MapReduce algorithms, due to their simplicity and practical efficiency.Comment: Conference version appears in SPAA 201