ANNz2: Photometric Redshift and Probability Distribution Function Estimation using Machine Learning

We present ANNz2, a new implementation of the public software for photometric redshift (photo-z) estimation of Collister & Lahav, which now includes generation of full probability distribution functions (PDFs). ANNz2 utilizes multiple machine learning methods, such as artificial neural networks and boosted decision/regression trees. The objective of the algorithm is to optimize the performance of the photo-z estimation, to properly derive the associated uncertainties, and to produce both single-value solutions and PDFs. In addition, estimators are made available, which mitigate possible problems of non-representative or incomplete spectroscopic training samples. ANNz2 has already been used as part of the first weak lensing analysis of the Dark Energy Survey, and is included in the experiment's first public data release. Here we illustrate the functionality of the code using data from the tenth data release of the Sloan Digital Sky Survey and the Baryon Oscillation Spectroscopic Survey. The code is available for download at http://github.com/IftachSadeh/ANNZ

A Joint Analysis for Cosmology and Photometric Redshift Calculation Using Cross Correlations

We present a method of calibrating the properties of photometric redshift bins as part of a larger Markov Chain Monte Carlo (MCMC) analysis for the inference of cosmological parameters. The redshift bins are characterised by their mean and variance, which are varied as free parameters and marginalised over when obtaining the cosmological parameters. We demonstrate that the likelihood function for cross-correlations in an angular power spectrum framework tightly constrains the properties of bins such that they may be well determined, reducing their influence on cosmological parameters and avoiding the bias from poorly estimated redshift distributions. We demonstrate that even with only three photometric and three spectroscopic bins, we can recover accurate estimates of the mean redshift of a bin to within $\Delta\mu \approx 3-4 \times10^{-3}$ and the width of the bin to $\Delta\sigma \approx 1\times10^{-3}$ for galaxies near $z = 1$. This indicates that we may be able to bring down the photometric redshift errors to a level which is in line with the requirements for the next generation of cosmological experiments

Radio weak lensing shear measurement in the visibility domain - I. Methodology

The high sensitivity of the new generation of radio telescopes such as the Square Kilometre Array (SKA) will allow cosmological weak lensing measurements at radio wavelengths that are competitive with optical surveys. We present an adaptation to radio data of lensfit, a method for galaxy shape measurement originally developed and used for optical weak lensing surveys. This likelihood method uses an analytical galaxy model and makes a Bayesian marginalization of the likelihood over uninteresting parameters. It has the feature of working directly in the visibility domain, which is the natural approach to adopt with radio interferometer data, avoiding systematics introduced by the imaging process. As a proof of concept, we provide results for visibility simulations of individual galaxies with flux density S ≥ 10 μJy at the phase centre of the proposed SKA1-MID baseline configuration, adopting 12 frequency channels in the band 950–1190 MHz. Weak lensing shear measurements from a population of galaxies with realistic flux and scalelength distributions are obtained after natural gridding of the raw visibilities. Shear measurements are expected to be affected by ‘noise bias’: we estimate the bias in the method as a function of signal-to-noise ratio (SNR). We obtain additive and multiplicative bias values that are comparable to SKA1 requirements for SNR > 18 and SNR > 30, respectively. The multiplicative bias for SNR >10 is comparable to that found in ground-based optical surveys such as CFHTLenS, and we anticipate that similar shear measurement calibration strategies to those used for optical surveys may be used to good effect in the analysis of SKA radio interferometer data

Upper Bound of 0.28 eV on Neutrino Masses from the Largest Photometric Redshift Survey

We present a new limit of Sigma m(nu) <= 0.28 (95% CL) on the sum of the neutrino masses assuming a flat Lambda CDM cosmology. This relaxes slightly to Sigma m(nu) <= 0.34 and Sigma m(nu) <= 0.47 when quasinonlinear scales are removed and w not equal -1, respectively. These are derived from a new photometric catalogue of over 700 000 luminous red galaxies (MegaZ DR7) with a volume of 3.3 (Gpc h(-1))(3) and redshift range 0.45 < z < 0.65. The data are combined with WMAP 5-year CMB, baryon acoustic oscillations, supernovae, and a Hubble Space Telescope prior on h. When combined with WMAP these data are as constraining as adding all supernovae and baryon oscillation data available. The upper limit is one of the tightest constraints on the neutrino from cosmology or particle physics. Further, if these bounds hold, they all predict that current-to-next generation neutrino experiments, such as KATRIN, are unlikely to obtain a detection

Degradation analysis in the estimation of photometric redshifts from non-representative training sets

We perform an analysis of photometric redshifts estimated by using a non-representative training sets in magnitude space. We use the ANNz2 and GPz algorithms to estimate the photometric redshift both in simulations and in real data from the Sloan Digital Sky Survey (DR12). We show that for the representative case, the results obtained by using both algorithms have the same quality, using either magnitudes or colours as input. In order to reduce the errors when estimating the redshifts with a non-representative training set, we perform the training in colour space. We estimate the quality of our results by using a mock catalogue which is split samples cuts in the r band between 19.4 < r < 20.8. We obtain slightly better results with GPz on single point z-phot estimates in the complete training set case, however the photometric redshifts estimated with ANNz2 algorithm allows us to obtain mildly better results in deeper r-band cuts when estimating the full redshift distribution of the sample in the incomplete training set case. By using a cumulative distribution function and a Monte Carlo process, we manage to define a photometric estimator which fits well the spectroscopic distribution of galaxies in the mock testing set, but with a larger scatter. To complete this work, we perform an analysis of the impact on the detection of clusters via density of galaxies in a field by using the photometric redshifts obtained with a non-representative training set

Evaluating machine learning techniques for predicting power spectra from reionization simulations

Upcoming experiments such as the SKA will provide huge quantities of data. Fast modelling of the high-redshift 21cm signal will be crucial for efficiently comparing these data sets with theory. The most detailed theoretical predictions currently come from numerical simulations and from faster but less accurate semi-numerical simulations. Recently, machine learning techniques have been proposed to emulate the behaviour of these semi-numerical simulations with drastically reduced time and computing cost. We compare the viability of five such machine learning techniques for emulating the 21cm power spectrum of the publicly-available code SimFast21. Our best emulator is a multilayer perceptron with three hidden layers, reproducing SimFast21 power spectra $10^8$ times faster than the simulation with 4% mean squared error averaged across all redshifts and input parameters. The other techniques (interpolation, Gaussian processes regression, and support vector machine) have slower prediction times and worse prediction accuracy than the multilayer perceptron. All our emulators can make predictions at any redshift and scale, which gives more flexible predictions but results in significantly worse prediction accuracy at lower redshifts. We then present a proof-of-concept technique for mapping between two different simulations, exploiting our best emulator's fast prediction speed. We demonstrate this technique to find a mapping between SimFast21 and another publicly-available code 21cmFAST. We observe a noticeable offset between the simulations for some regions of the input space. Such techniques could potentially be used as a bridge between fast semi-numerical simulations and accurate numerical radiative transfer simulations

Radio galaxy detection in the visibility domain

We explore a new Bayesian method of detecting galaxies from radio interferometric data of the faint sky. Working in the Fourier domain, we fit a single, parameterized galaxy model to simulated visibility data of star-forming galaxies. The resulting multimodal posterior distribution is then sampled using a multimodal nested sampling algorithm such as MULTINEST. For each galaxy, we construct parameter estimates for the position, flux, scale length, and ellipticities from the posterior samples. We first test our approach on simulated SKA1-MID visibility data of up to 100 galaxies in the field of view (FOV), considering a typical weak lensing survey regime (SNR ≥ 10) where 98 per cent of the input galaxies are detected with no spurious source detections. We then explore the low-SNR regime, finding our approach reliable in galaxy detection and providing in particular high accuracy in positional estimates down to SNR ∼ 5. The presented method does not require transformation of visibilities to the image domain, and requires no prior knowledge of the number of galaxies in the FOV, thus could become a useful tool for constructing accurate radio galaxy catalogues in the future

Radio galaxy shape measurement with Hamiltonian Monte Carlo in the visibility domain

Radio weak lensing, while a highly promising complementary probe to optical weak lensing, will require incredible precision in the measurement of galaxy shape parameters. In this paper, we extend the Bayesian Inference for Radio Observations model fitting approach to measure galaxy shapes directly from visibility data of radio continuum surveys, instead of from image data. We apply a Hamiltonian Monte Carlo (HMC) technique for sampling the posterior, which is more efficient than the standard Monte Carlo Markov Chain method when dealing with a large dimensional parameter space. Adopting the exponential profile for galaxy model fitting allows us to analytically calculate the likelihood gradient required by HMC, allowing a faster and more accurate sampling. The method is tested on SKA1-MID simulated observations at 1.4 GHz of a field containing up to 1000 star-forming galaxies. It is also applied to a simulated observation of the weak lensing precursor survey SuperCLASS. In both cases we obtain reliable measurements of the galaxies' ellipticity and size for all sources with signal-to-noise ratio ≥ 10, and we also find relationships between the convergence properties of the HMCtechnique and some source parameters. Direct shape measurement in the visibility domain achieves high accuracy at the expected source number densities of the current and next SKA precursor continuum surveys. The proposed method can be easily extended for the fitting of other galaxy and scientific parameters, as well as simultaneously marginalizing over systematic and instrumental effects

Simulations of systematic direction-dependent instrumental effects in intensity mapping experiments

Intensity mapping experiment treats the 21 cm radio emission as a diffuse source and allows smaller and relatively cheaper radio antennas with short baselines to be used in such experiments. However, the technique is restricted by the precise subtraction of the foreground continuum signal from Galactic and extragalactic radio sources. Furthermore, the signal is subjected to direction-dependent effects, particularly the primary beam, as it modulates the intensity as a function of the sky position. In addition, due to the imperfections in the antenna feeds, a portion of the polarized foreground tends to find its way into the total intensity, making it a major obstacle to detect the H I signal. In the case of dish arrays, this will be dominated by the instrument mispointings and polarization leakage. To estimate this contamination, we use OSKAR to simulate ‘dish-like’ primary beams and then perturb these primary beams by introducing gain, phase, and surface distribution errors. We then simulate the foregrounds with these modelled beams to determine the errors in Stokes I and also observe the amount of |Q + iU| that corrupts I. Our simulation shows that the H I signal power can be measured at a multipole moment of l = 100 if we do not correct for any polarization leakage of the beam and at a multiple moment of l = 25 if we correct for the beam from I, assuming the beam is not known to the extent to which we have considered in this paper

Foreground Subtraction in Intensity Mapping with the SKA

21cm intensity mapping experiments aim to observe the diffuse neutral hydrogen (HI) distribution on large scales which traces the Cosmic structure. The Square Kilometre Array (SKA) will have the capacity to measure the 21cm signal over a large fraction of the sky. However, the redshifted 21cm signal in the respective frequencies is faint compared to the Galactic foregrounds produced by synchrotron and free-free electron emission. In this article, we review selected foreground subtraction methods suggested to effectively separate the 21cm signal from the foregrounds with intensity mapping simulations or data. We simulate an intensity mapping experiment feasible with SKA phase 1 including extragalactic and Galactic foregrounds. We give an example of the residuals of the foreground subtraction with a independent component analysis and show that the angular power spectrum is recovered within the statistical errors on most scales. Additionally, the scale of the Baryon Acoustic Oscillations is shown to be unaffected by foreground subtraction