Solving high-dimensional parameter inference: marginal posterior densities & Moment Networks

High-dimensional probability density estimation for inference suffers from the "curse of dimensionality". For many physical inference problems, the full posterior distribution is unwieldy and seldom used in practice. Instead, we propose direct estimation of lower-dimensional marginal distributions, bypassing high-dimensional density estimation or high-dimensional Markov chain Monte Carlo (MCMC) sampling. By evaluating the two-dimensional marginal posteriors we can unveil the full-dimensional parameter covariance structure. We additionally propose constructing a simple hierarchy of fast neural regression models, called Moment Networks, that compute increasing moments of any desired lower-dimensional marginal posterior density; these reproduce exact results from analytic posteriors and those obtained from Masked Autoregressive Flows. We demonstrate marginal posterior density estimation using high-dimensional LIGO-like gravitational wave time series and describe applications for problems of fundamental cosmology

Cosmic shear: Inference from forward models

Density-estimation likelihood-free inference (DELFI) has recently been proposed as an efficient method for simulation-based cosmological parameter inference. Compared to the standard likelihood-based Markov chain Monte Carlo (MCMC) approach, DELFI has several advantages: it is highly parallelizable, there is no need to assume a possibly incorrect functional form for the likelihood, and complicated effects (e.g., the mask and detector systematics) are easier to handle with forward models. In light of this, we present two DELFI pipelines to perform weak lensing parameter inference with log-normal realizations of the tomographic shear field—using the C_{l} summary statistic. The first pipeline accounts for the non-Gaussianities of the shear field, intrinsic alignments, and photometric-redshift error. We validate that it is accurate enough for Stage III experiments and estimate that sigma (1000) simulations are needed to perform inference on Stage IV data. By comparing the second DELFI pipeline, which makes no assumption about the functional form of the likelihood, with the standard MCMC approach, which assumes a Gaussian likelihood, we test the impact of the Gaussian likelihood approximation in the MCMC analysis. We find it has a negligible impact on Stage IV parameter constraints. Our pipeline is a step towards seamlessly propagating all data-processing, instrumental, theoretical, and astrophysical systematics through to the final parameter constraints

Bayesian astrostatistics: a backward look to the future

This perspective chapter briefly surveys: (1) past growth in the use of Bayesian methods in astrophysics; (2) current misconceptions about both frequentist and Bayesian statistical inference that hinder wider adoption of Bayesian methods by astronomers; and (3) multilevel (hierarchical) Bayesian modeling as a major future direction for research in Bayesian astrostatistics, exemplified in part by presentations at the first ISI invited session on astrostatistics, commemorated in this volume. It closes with an intentionally provocative recommendation for astronomical survey data reporting, motivated by the multilevel Bayesian perspective on modeling cosmic populations: that astronomers cease producing catalogs of estimated fluxes and other source properties from surveys. Instead, summaries of likelihood functions (or marginal likelihood functions) for source properties should be reported (not posterior probability density functions), including nontrivial summaries (not simply upper limits) for candidate objects that do not pass traditional detection thresholds.Comment: 27 pp, 4 figures. A lightly revised version of a chapter in "Astrostatistical Challenges for the New Astronomy" (Joseph M. Hilbe, ed., Springer, New York, forthcoming in 2012), the inaugural volume for the Springer Series in Astrostatistics. Version 2 has minor clarifications and an additional referenc

Bayesian analysis of cosmic structures

We revise the Bayesian inference steps required to analyse the cosmological large-scale structure. Here we make special emphasis in the complications which arise due to the non-Gaussian character of the galaxy and matter distribution. In particular we investigate the advantages and limitations of the Poisson-lognormal model and discuss how to extend this work. With the lognormal prior using the Hamiltonian sampling technique and on scales of about 4 h^{-1} Mpc we find that the over-dense regions are excellent reconstructed, however, under-dense regions (void statistics) are quantitatively poorly recovered. Contrary to the maximum a posteriori (MAP) solution which was shown to over-estimate the density in the under-dense regions we obtain lower densities than in N-body simulations. This is due to the fact that the MAP solution is conservative whereas the full posterior yields samples which are consistent with the prior statistics. The lognormal prior is not able to capture the full non-linear regime at scales below ~ 10 h^{-1} Mpc for which higher order correlations would be required to describe the matter statistics. However, we confirm as it was recently shown in the context of Ly-alpha forest tomography that the Poisson-lognormal model provides the correct two-point statistics (or power-spectrum).Comment: 11 pages, 1 figure, report for the Astrostatistics and Data Mining workshop, La Palma, Spain, 30 May - 3 June 2011, to appear in Springer Series on Astrostatistic

Planck 2018 results. VII. Isotropy and Statistics of the CMB

Analysis of the Planck 2018 data set indicates that the statistical properties of the cosmic microwave background (CMB) temperature anisotropies are in excellent agreement with previous studies using the 2013 and 2015 data releases. In particular, they are consistent with the Gaussian predictions of the $\Lambda$CDM cosmological model, yet also confirm the presence of several so-called "anomalies" on large angular scales. The novelty of the current study, however, lies in being a first attempt at a comprehensive analysis of the statistics of the polarization signal over all angular scales, using either maps of the Stokes parameters, $Q$ and $U$, or the $E$-mode signal derived from these using a new methodology (which we describe in an appendix). Although remarkable progress has been made in reducing the systematic effects that contaminated the 2015 polarization maps on large angular scales, it is still the case that residual systematics (and our ability to simulate them) can limit some tests of non-Gaussianity and isotropy. However, a detailed set of null tests applied to the maps indicates that these issues do not dominate the analysis on intermediate and large angular scales (i.e., $\ell \lesssim 400$). In this regime, no unambiguous detections of cosmological non-Gaussianity, or of anomalies corresponding to those seen in temperature, are claimed. Notably, the stacking of CMB polarization signals centred on the positions of temperature hot and cold spots exhibits excellent agreement with the $\Lambda$CDM cosmological model, and also gives a clear indication of how Planck provides state-of-the-art measurements of CMB temperature and polarization on degree scales

Planck 2018 results. VII. Isotropy and statistics of the CMB

Analysis of the Planck 2018 data set indicates that the statistical properties of the cosmic microwave background (CMB) temperature anisotropies are in excellent agreement with previous studies using the 2013 and 2015 data releases. In particular, they are consistent with the Gaussian predictions of the $\Lambda$CDM cosmological model, yet also confirm the presence of several so-called "anomalies" on large angular scales. The novelty of the current study, however, lies in being a first attempt at a comprehensive analysis of the statistics of the polarization signal over all angular scales, using either maps of the Stokes parameters, $Q$ and $U$, or the $E$-mode signal derived from these using a new methodology (which we describe in an appendix). Although remarkable progress has been made in reducing the systematic effects that contaminated the 2015 polarization maps on large angular scales, it is still the case that residual systematics (and our ability to simulate them) can limit some tests of non-Gaussianity and isotropy. However, a detailed set of null tests applied to the maps indicates that these issues do not dominate the analysis on intermediate and large angular scales (i.e., $\ell \lesssim 400$). In this regime, no unambiguous detections of cosmological non-Gaussianity, or of anomalies corresponding to those seen in temperature, are claimed. Notably, the stacking of CMB polarization signals centred on the positions of temperature hot and cold spots exhibits excellent agreement with the $\Lambda$CDM cosmological model, and also gives a clear indication of how Planck provides state-of-the-art measurements of CMB temperature and polarization on degree scales

Planck intermediate results. XIX. An overview of the polarized thermal emission from Galactic dust

This paper presents an overview of the polarized sky as seen by Planck HFI at 353 GHz, which is the most sensitive Planck channel for dust polarization. We construct and analyse maps of dust polarization fraction and polarization angle at 1° resolution, taking into account noise bias and possible systematic effects. The sensitivity of the Planck HFI polarization measurements allows for the first time a mapping of Galactic dust polarized emission on large scales, including low column density regions. We find that the maximum observed dust polarization fraction is high (pmax = 19.8%), in particular in some regions of moderate hydrogen column density (NH < 2 × 1021 cm-2). The polarization fraction displays a large scatter at NH below a few 1021 cm-2. There is a general decrease in the dust polarization fraction with increasing column density above NH ≃ 1 × 1021 cm-2 and in particular a sharp drop above NH ≃ 1.5 × 1022 cm-2. We characterize the spatial structure of the polarization angle using the angle dispersion function. We find that the polarization angle is ordered over extended areas of several square degrees, separated by filamentary structures of high angle dispersion function. These appear as interfaces where the sky projection of the magnetic field changes abruptly without variations in the column density. The polarization fraction is found to be anti-correlated with the dispersion of polarization angles. These results suggest that, at the resolution of 1°, depolarization is due mainly to fluctuations in the magnetic field orientation along the line of sight, rather than to the loss of grain alignment in shielded regions. We also compare the polarization of thermal dust emission with that of synchrotron measured with Planck, low-frequency radio data, and Faraday rotation measurements toward extragalactic sources. These components bear resemblance along the Galactic plane and in some regions such as the Fan and North Polar Spur regions. The poor match observed in other regions shows, however, that dust, cosmic-ray electrons, and thermal electrons generally sample different parts of the line of sight. Reproduced with permission, © ESO, 201

Snowmass2021 - Letter of interest cosmology intertwined I: Perspectives for the next decade

The standard Cold Dark Matter cosmological model provides an amazing description of a wide range of astrophysical and astronomical data. However, there are a few big open questions, that make the standard model look like a first-order approximation to a more realistic scenario that still needs to be fully understood. In this Letter of Interest we will list a few important goals that need to be addressed in the next decade, also taking into account the current discordances present between the different cosmological probes, as the Hubble constant value, the tension, and the anomalies present in the Planck results. Finally, we will give an overview of upgraded experiments and next-generation space-missions and facilities on Earth that will be of crucial importance to address all these questions

Snowmass2021 - Letter of interest cosmology intertwined IV: the age of the universe and its curvature

A precise measurement of the curvature of the Universe is of prime importance for cosmology since it could not only confirm the paradigm of primordial inflation but also help in discriminating between different early-Universe scenarios. Recent observations, while broadly consistent with a spatially flat standard Cold Dark Matter (CDM) model, show tensions that still allow (and, in some cases, even suggest) a few percent deviations from a flat universe. In particular, the Planck Cosmic Microwave Background power spectra, assuming the nominal likelihood, prefer a closed universe at more than 99% confidence level. While new physics could be at play, this anomaly may be the result of an unresolved systematic error or just a statistical fluctuation. However, since positive curvature allows a larger age of the Universe, an accurate determination of the age of the oldest objects provides a smoking gun in confirming or falsifying the current flat CDM model

Cosmology intertwined III: fσ8 and S8

The standard Cold Dark Matter cosmological model provides a wonderful fit to current cosmological data, but a few statistically significant tensions and anomalies were found in the latest data analyses. While these anomalies could be due to the presence of systematic errors in the experiments, they could also indicate the need for new physics beyond the standard model. In this Letter of Interest we focus on the tension between Planck data and weak lensing measurements and redshift surveys, in the value of the matter energy density and the amplitude (or the growth rate ) of cosmic structure. We list a few promising models for solving this tension, and discuss the importance of trying to fit multiple cosmological datasets with complete physical models, rather than fitting individual datasets with a few handpicked theoretical parameters