Photometric Redshift Calibration with Self Organising Maps

Accurate photometric redshift calibration is central to the robustness of all cosmology constraints from cosmic shear surveys. Analyses of the KiDS re-weighted training samples from all overlapping spectroscopic surveys to provide a direct redshift calibration. Using self-organising maps (SOMs) we demonstrate that this spectroscopic compilation is sufficiently complete for KiDS, representing $99\%$ of the effective 2D cosmic shear sample. We use the SOM to define a $100\%$ represented `gold' cosmic shear sample, per tomographic bin. Using mock simulations of KiDS and the spectroscopic training set, we estimate the uncertainty on the SOM redshift calibration, and find that photometric noise, sample variance, and spectroscopic selection effects (including redshift and magnitude incompleteness) induce a combined maximal scatter on the bias of the redshift distribution reconstruction ($\Delta \langle z \rangle=\langle z \rangle_{\rm est}-\langle z \rangle_{\rm true}$) of $\sigma_{\Delta \langle z \rangle} \leq 0.006$ in all tomographic bins. We show that the SOM calibration is unbiased in the cases of noiseless photometry and perfectly representative spectroscopic datasets, as expected from theory. The inclusion of both photometric noise and spectroscopic selection effects in our mock data introduces a maximal bias of $\Delta \langle z \rangle =0.013\pm0.006$, or $\Delta \langle z \rangle \leq 0.025$ at $97.5\%$ confidence, once quality flags have been applied to the SOM. The method presented here represents a significant improvement over the previously adopted direct redshift calibration implementation for KiDS, owing to its diagnostic and quality assurance capabilities. The implementation of this method in future cosmic shear studies will allow better diagnosis, examination, and mitigation of systematic biases in photometric redshift calibration.Comment: 22 pages, 10 figures, 4 appendices, accepted for publication in A&

KiDS+VIKING-450:Improved cosmological parameter constraints from redshift calibration with self-organising maps

We present updated cosmological constraints for the KiDS+VIKING-450 cosmic shear data set (KV450), estimated using redshift distributions and photometric samples defined using self-organising maps (SOMs). Our fiducial analysis finds marginal posterior constraints of $S_8\equiv\sigma_8\sqrt{\Omega_{\rm m}/0.3}=0.716^{+0.043}_{-0.038}$; smaller than, but otherwise consistent with, previous work using this data set ($|\Delta S_8| = 0.023$). We analyse additional samples and redshift distributions constructed in three ways: excluding certain spectroscopic surveys during redshift calibration, excluding lower-confidence spectroscopic redshifts in redshift calibration, and considering only photometric sources which are jointly calibrated by at least three spectroscopic surveys. In all cases, the method utilised here proves robust: we find a maximal deviation from our fiducial analysis of $|\Delta S_8| \leq 0.011$ for all samples defined and analysed using our SOM. To demonstrate the reduction in systematic biases found within our analysis, we highlight our results when performing redshift calibration without the DEEP2 spectroscopic data set. In this case we find marginal posterior constraints of $S_8=0.707_{-0.042}^{+0.046}$; a difference with respect to the fiducial that is both significantly smaller than, and in the opposite direction to, the equivalent shift from previous work. These results suggest that our improved cosmological parameter estimates are insensitive to pathological misrepresentation of photometric sources by the spectroscopy used for direct redshift calibration, and therefore that this systematic effect cannot be responsible for the observed difference between $S_8$ estimates made with KV450 and Planck CMB probes.Comment: 10 pages, 3 figures, 4 appendices, accepted for publication in A&A Letter

The effects of varying depth in cosmic shear surveys

We present a semi-analytic model for the shear two-point correlation function of a cosmic shear survey with non-uniform depth. Ground-based surveys are subject to depth variations that primarily arise through varying atmospheric conditions. For a survey like the Kilo-Degree Survey (KiDS), we find that the measured depth variation increases the amplitude of the observed shear correlation function at the level of a few percent out to degree-scales, relative to the assumed uniform-depth case. The impact on the inferred cosmological parameters is shown to be insignificant for a KiDS-like survey. For next-generation cosmic shear experiments, however, we conclude that variable depth should be accounted for

Strong Lensing Model of SPT-CLJ0356-5337, a Major Merger Candidate at Redshift 1.0359

We present an analysis of the mass distribution inferred from strong lensing by SPT-CL J0356-5337, a cluster of galaxies at redshift z = 1.0359 revealed in the follow-up of the SPT-SZ clusters. The cluster has an Einstein radius of Erad=14 for a source at z = 3 and a mass within 500 kpc of M_500kpc = 4.0+-0.8x10^14Msol. Our spectroscopic identification of three multiply-imaged systems (z = 2.363, z = 2.364, and z = 3.048), combined with HST F606W-band imaging allows us to build a strong lensing model for this cluster with an rms of <0.3'' between the predicted and measured positions of the multiple images. Our modeling reveals a two-component mass distribution in the cluster. One mass component is dominated by the brightest cluster galaxy and the other component, separated by ~170 kpc, contains a group of eight red elliptical galaxies confined in a ~9'' (~70 kpc) diameter circle. We estimate the mass ratio between the two components to be between 1:1.25 and 1:1.58. In addition, spectroscopic data reveal that these two near-equal mass cores have only a small velocity difference of 300 km/s between the two components. This small radial velocity difference suggests that most of the relative velocity takes place in the plane of the sky, and implies that SPT-CL J0356-5337 is a major merger with a small impact parameter seen face-on. We also assess the relative contributions of galaxy-scale halos to the overall mass of the core of the cluster and find that within 800 kpc from the brightest cluster galaxy about 27% of the total mass can be attributed to visible and dark matter associated with galaxies, whereas only 73% of the total mass in the core comes from cluster-scale dark matter halos.Comment: 19 pages, 11 figures. Submitted to Ap

Magnification bias in galaxy surveys with complex sample selection functions

Gravitational lensing magnification modifies the observed spatial distribution of galaxies and can severely bias cosmological probes of large-scale structure if not accurately modelled. Standard approaches to modelling this magnification bias may not be applicable in practice as many galaxy samples have complex, often implicit, selection functions. We propose and test a procedure to quantify the magnification bias induced in clustering and galaxy-galaxy lensing (GGL) signals in galaxy samples subject to a selection function beyond a simple flux limit. The method employs realistic mock data to calibrate an effective luminosity function slope, $\alpha_{\rm{obs}}$, from observed galaxy counts, which can then be used with the standard formalism. We demonstrate this method for two galaxy samples derived from the Baryon Oscillation Spectroscopic Survey (BOSS) in the redshift ranges $0.2 < z \leq 0.5$ and $0.5 < z \leq 0.75$, complemented by mock data built from the MICE2 simulation. We obtain $\alpha_{\rm{obs}} = 1.93 \pm 0.05$ and $\alpha_{\rm{obs}} = 2.62 \pm 0.28$ for the two BOSS samples. For BOSS-like lenses, we forecast a contribution of the magnification bias to the GGL signal between the angular scales of $100$ and $4600$ with a cumulative signal-to-noise ratio between $0.1$ and $1.1$ for sources from the Kilo-Degree Survey (KiDS), between $0.4$ and $2.0$ for sources from the Hyper Suprime-Cam survey (HSC), and between $0.3$ and $2.8$ for ESA Euclid-like source samples. These contributions are significant enough to require explicit modelling in future analyses of these and similar surveys.Comment: 15 pages, 13 figure

KiDS-Legacy calibration: unifying shear and redshift calibration with the SKiLLS multi-band image simulations

We present SKiLLS, a suite of multi-band image simulations for the weak lensing analysis of the complete Kilo-Degree Survey (KiDS), dubbed KiDS-Legacy analysis. The resulting catalogues enable joint shear and redshift calibration, enhancing the realism and hence accuracy over previous efforts. To create a large volume of simulated galaxies with faithful properties and to a sufficient depth, we integrated cosmological simulations with high-quality imaging observations. We also improved the realism of simulated images by allowing the point spread function (PSF) to differ between CCD images, including stellar density variations and varying noise levels between pointings. Using realistic variable shear fields, we accounted for the impact of blended systems at different redshifts. Although the overall correction is minor, we found a clear redshift-bias correlation in the blending-only variable shear simulations, indicating the non-trivial impact of this higher-order blending effect. We also explored the impact of the PSF modelling errors and found a small yet noticeable effect on the shear bias. Finally, we conducted a series of sensitivity tests, including changing the input galaxy properties. We conclude that our fiducial shape measurement algorithm, lensfit, is robust within the requirements of lensing analyses with KiDS. As for future weak lensing surveys with tighter requirements, we suggest further investments in understanding the impact of blends at different redshifts, improving the PSF modelling algorithm and developing the shape measurement method to be less sensitive to the galaxy properties.Comment: 28 pages, 31 figures, 2 tables, minor revisions to match the final accepted versio

KiDS-1000: cross-correlation with Planck cosmic microwave background lensing and intrinsic alignment removal with self-calibration

Galaxy shear - cosmic microwave background (CMB) lensing convergence cross-correlations contain additional information on cosmology to auto-correlations. While being immune to certain systematic effects, they are affected by the galaxy intrinsic alignments (IA). This may be responsible for the reported low lensing amplitude of the galaxy shear $\times$ CMB convergence cross-correlations, compared to the standard Planck $\Lambda$CDM (cosmological constant and cold dark matter) cosmology prediction. In this work, we investigate how IA affects the Kilo-Degree Survey (KiDS) galaxy lensing shear - Planck CMB lensing convergence cross-correlation and compare it to previous treatments with or without IA taken into consideration. More specifically, we compare marginalization over IA parameters and the IA self-calibration (SC) method (with additional observables defined only from the source galaxies) and prove that SC can efficiently break the degeneracy between the CMB lensing amplitude $A_{\rm lens}$ and the IA amplitude $A_{\rm IA}$. We further investigate how different systematics affect the resulting $A_{\rm IA}$ and $A_{\rm lens}$, and validate our results with the MICE2 simulation. We find that by including the SC method to constrain IA, the information loss due to the degeneracy between CMB lensing and IA is strongly reduced. The best-fit values are $A_{\rm lens}=0.84^{+0.22}_{-0.22}$ and $A_{\rm IA}=0.60^{+1.03}_{-1.03}$, while different angular scale cuts can affect $A_{\rm lens}$ by $\sim10\%$. We show that appropriate treatment of the boost factor, cosmic magnification, and photometric redshift modeling is important for obtaining the correct IA and cosmological results.Comment: match version accepted by A&

KiDS-1000: Cosmology with improved cosmic shear measurements

We present refined cosmological parameter constraints derived from a cosmic shear analysis of the fourth data release of the Kilo-Degree Survey (KiDS-1000). Our main improvements include enhanced galaxy shape measurements made possible by an updated version of the lensfit code and improved shear calibration achieved with a newly developed suite of multi-band image simulations. Additionally, we incorporated recent advancements in cosmological inference from the joint Dark Energy Survey Year 3 and KiDS-1000 cosmic shear analysis. Assuming a spatially flat standard cosmological model, we constrain $S_8\equiv\sigma_8(\Omega_{\rm m}/0.3)^{0.5} = 0.776_{-0.027-0.003}^{+0.029+0.002}$, where the second set of uncertainties accounts for the systematic uncertainties within the shear calibration. These systematic uncertainties stem from minor deviations from realism in the image simulations and the sensitivity of the shear measurement algorithm to the morphology of the galaxy sample. Despite these changes, our results align with previous KiDS studies and other weak lensing surveys, and we find a ${\sim}2.3\sigma$ level of tension with the Planck cosmic microwave background constraints on $S_8$.Comment: 20 pages, 13 figures, 4 tables, minor revisions to match the final accepted versio

KiDS+VIKING-450 and DES-Y1 combined::Mitigating baryon feedback uncertainty with COSEBIs

We present cosmological constraints from a joint cosmic shear analysis of the Kilo-Degree Survey (KV450) and the Dark Energy Survey (DES-Y1), conducted using Complete Orthogonal Sets of E/B-Integrals (COSEBIs). With COSEBIs we isolate any B-modes which have a non-cosmic shear origin and demonstrate the robustness of our cosmological E-mode analysis as no significant B-modes are detected. We highlight how COSEBIs are fairly insensitive to the amplitude of the non-linear matter power spectrum at high $k$-scales, mitigating the uncertain impact of baryon feedback in our analysis. COSEBIs, therefore, allow us to utilise additional small-scale information, improving the DES-Y1 joint constraints on $S_8=\sigma_8(\Omega_{\rm m}/0.3)^{0.5}$ and $\Omega_{\rm m}$ by $20\%$. Adopting a flat $\Lambda$CDM model we find $S_8=0.755^{+0.019}_{-0.021}$, which is in $3.2\sigma$ tension with the Planck Legacy analysis of the cosmic microwave background.Comment: Accepted for publication in A&A. 15 pages, 7 figure

KiDS-1000 Cosmology:Multi-probe weak gravitational lensing and spectroscopic galaxy clustering constraints

We present a joint cosmological analysis of weak gravitational lensing observations from the Kilo-Degree Survey (KiDS-1000), with redshift-space galaxy clustering observations from the Baryon Oscillation Spectroscopic Survey (BOSS), and galaxy-galaxy lensing observations from the overlap between KiDS-1000, BOSS and the spectroscopic 2-degree Field Lensing Survey (2dFLenS). This combination of large-scale structure probes breaks the degeneracies between cosmological parameters for individual observables, resulting in a constraint on the structure growth parameter $S_8=\sigma_8 \sqrt{\Omega_{\rm m}/0.3} = 0.766^{+0.020}_{-0.014}$, that has the same overall precision as that reported by the full-sky cosmic microwave background observations from Planck. The recovered $S_8$ amplitude is low, however, by $8.3 \pm 2.6$ % relative to Planck. This result builds from a series of KiDS-1000 analyses where we validate our methodology with variable depth mock galaxy surveys, our lensing calibration with image simulations and null-tests, and our optical-to-near-infrared redshift calibration with multi-band mock catalogues and a spectroscopic-photometric clustering analysis. The systematic uncertainties identified by these analyses are folded through as nuisance parameters in our cosmological analysis. Inspecting the offset between the marginalised posterior distributions, we find that the $S_8$-difference with Planck is driven by a tension in the matter fluctuation amplitude parameter, $\sigma_8$. We quantify the level of agreement between the CMB and our large-scale structure constraints using a series of different metrics, finding differences with a significance ranging between $\sim\! 3\,\sigma$, when considering the offset in $S_{8}$, and $\sim\! 2\,\sigma$, when considering the full multi-dimensional parameter space.Comment: 25 pages, 11 figures, 5 tables, A&A accepted, including a new appendix on Intrinsic Alignments. The KiDS-1000 data products are available for download at http://kids.strw.leidenuniv.nl/DR4/lensing.php. This data release includes open source software, the shear-photo-z catalogue, the cosmic shear and 3x2pt data vectors and covariances, and posteriors in the form of Multinest chain