Euclid preparation. XXVI. The Euclid Morphology Challenge: Towards structural parameters for billions of galaxies

The various Euclid imaging surveys will become a reference for studies of galaxy morphology by delivering imaging over an unprecedented area of 15 000 square degrees with high spatial resolution. In order to understand the capabilities of measuring morphologies from Euclid-detected galaxies and to help implement measurements in the pipeline of the Organisational Unit MER of the Euclid Science Ground Segment, we have conducted the Euclid Morphology Challenge, which we present in two papers. While the companion paper focusses on the analysis of photometry, this paper assesses the accuracy of the parametric galaxy morphology measurements in imaging predicted from within the Euclid Wide Survey. We evaluate the performance of five state-of-the-art surface-brightness-fitting codes, , , , and , on a sample of about 1.5 million simulated galaxies (350 000 above 5σ) resembling reduced observations with the Euclid VIS and NIR instruments. The simulations include analytic Sérsic profiles with one and two components, as well as more realistic galaxies generated with neural networks. We find that, despite some code-specific differences, all methods tend to achieve reliable structural measurements (< 10% scatter on ideal Sérsic simulations) down to an apparent magnitude of about I$_{E}$ = 23 in one component and I$_{E}$ = 21 in two components, which correspond to a signal-to-noise ratio of approximately 1 and 5, respectively. We also show that when tested on non-analytic profiles, the results are typically degraded by a factor of 3, driven by systematics. We conclude that the official Euclid Data Releases will deliver robust structural parameters for at least 400 million galaxies in the Euclid Wide Survey by the end of the mission. We find that a key factor for explaining the different behaviour of the codes at the faint end is the set of adopted priors for the various structural parameters

Euclid preparation: XXVI. the Euclid Morphology Challenge: Towards structural parameters for billions of galaxies

The various Euclid imaging surveys will become a reference for studies of galaxy morphology by delivering imaging over an unprecedented area of 15 000 square degrees with high spatial resolution. In order to understand the capabilities of measuring morphologies from Euclid-detected galaxies and to help implement measurements in the pipeline of the Organisational Unit MER of the Euclid Science Ground Segment, we have conducted the Euclid Morphology Challenge, which we present in two papers. While the companion paper focusses on the analysis of photometry, this paper assesses the accuracy of the parametric galaxy morphology measurements in imaging predicted from within the Euclid Wide Survey. We evaluate the performance of five state-of-the-art surface-brightness-fitting codes, DeepLeGATo, Galapagos-2, Morfometryka, ProFit and SourceXtractor++, on a sample of about 1.5 million simulated galaxies (350 000 above 5σ) resembling reduced observations with the Euclid VIS and NIR instruments. The simulations include analytic Sérsic profiles with one and two components, as well as more realistic galaxies generated with neural networks. We find that, despite some code-specific differences, all methods tend to achieve reliable structural measurements (< 10% scatter on ideal Sérsic simulations) down to an apparent magnitude of about IE = 23 in one component and IE = 21 in two components, which correspond to a signal-to-noise ratio of approximately 1 and 5, respectively. We also show that when tested on non-analytic profiles, the results are typically degraded by a factor of 3, driven by systematics. We conclude that the official Euclid Data Releases will deliver robust structural parameters for at least 400 million galaxies in the Euclid Wide Survey by the end of the mission. We find that a key factor for explaining the different behaviour of the codes at the faint end is the set of adopted priors for the various structural parameters

Euclid preparation. XXV. The Euclid Morphology Challenge -- Towards model-fitting photometry for billions of galaxies

The ESA Euclid mission will provide high-quality imaging for about 1.5 billion galaxies. A software pipeline to automatically process and analyse such a huge amount of data in real time is being developed by the Science Ground Segment of the Euclid Consortium; this pipeline will include a model-fitting algorithm, which will provide photometric and morphological estimates of paramount importance for the core science goals of the mission and for legacy science. The Euclid Morphology Challenge is a comparative investigation of the performance of five model-fitting software packages on simulated Euclid data, aimed at providing the baseline to identify the best suited algorithm to be implemented in the pipeline. In this paper we describe the simulated data set, and we discuss the photometry results. A companion paper (Euclid Collaboration: Bretonni\`ere et al. 2022) is focused on the structural and morphological estimates. We created mock Euclid images simulating five fields of view of 0.48 deg2 each in the $I_E$ band of the VIS instrument, each with three realisations of galaxy profiles (single and double S\'ersic, and 'realistic' profiles obtained with a neural network); for one of the fields in the double S\'ersic realisation, we also simulated images for the three near-infrared $Y_E$, $J_E$ and $H_E$ bands of the NISP-P instrument, and five Rubin/LSST optical complementary bands ($u$, $g$, $r$, $i$, and $z$). To analyse the results we created diagnostic plots and defined ad-hoc metrics. Five model-fitting software packages (DeepLeGATo, Galapagos-2, Morfometryka, ProFit, and SourceXtractor++) were compared, all typically providing good results. (cut)Comment: 29 pages, 33 figures. Euclid pre-launch key paper. Companion paper: Bretonniere et al. 202

Euclid preparation XXVI. The Euclid Morphology Challenge. Towards structural parameters for billions of galaxies

The various Euclid imaging surveys will become a reference for studies of galaxy morphology by delivering imaging over an unprecedented area of 15 000 square degrees with high spatial resolution. In order to understand the capabilities of measuring morphologies from Euclid-detected galaxies and to help implement measurements in the pipeline, we have conducted the Euclid Morphology Challenge, which we present in two papers. While the companion paper by Merlin et al. focuses on the analysis of photometry, this paper assesses the accuracy of the parametric galaxy morphology measurements in imaging predicted from within the Euclid Wide Survey. We evaluate the performance of five state-of-the-art surface-brightness-fitting codes DeepLeGATo, Galapagos-2, Morfometryka, Profit and SourceXtractor++ on a sample of about 1.5 million simulated galaxies resembling reduced observations with the Euclid VIS and NIR instruments. The simulations include analytic S\'ersic profiles with one and two components, as well as more realistic galaxies generated with neural networks. We find that, despite some code-specific differences, all methods tend to achieve reliable structural measurements (10% scatter on ideal S\'ersic simulations) down to an apparent magnitude of about 23 in one component and 21 in two components, which correspond to a signal-to-noise ratio of approximately 1 and 5 respectively. We also show that when tested on non-analytic profiles, the results are typically degraded by a factor of 3, driven by systematics. We conclude that the Euclid official Data Releases will deliver robust structural parameters for at least 400 million galaxies in the Euclid Wide Survey by the end of the mission. We find that a key factor for explaining the different behaviour of the codes at the faint end is the set of adopted priors for the various structural parameters.Comment: Accepted by A&A. 30 pages, 23+6 figures, Euclid pre-launch key paper. Companion paper: Euclid Collaboration XXV: Merlin et al. 2022 Minor corrections after journal revie

Euclid preparation: X. The Euclid photometric-redshift challenge

Forthcoming large photometric surveys for cosmology require precise and accurate photometric redshift (photo-z) measurements for the success of their main science objectives. However, to date, no method has been able to produce photo-zs at the required accuracy using only the broad-band photometry that those surveys will provide. An assessment of the strengths and weaknesses of current methods is a crucial step in the eventual development of an approach to meet this challenge. We report on the performance of 13 photometric redshift code single value redshift estimates and redshift probability distributions (PDZs) on a common set of data, focusing particularly on the 0.2−2.6 redshift range that the Euclid mission will probe. We designed a challenge using emulated Euclid data drawn from three photometric surveys of the COSMOS field. The data was divided into two samples: one calibration sample for which photometry and redshifts were provided to the participants; and the validation sample, containing only the photometry to ensure a blinded test of the methods. Participants were invited to provide a redshift single value estimate and a PDZ for each source in the validation sample, along with a rejection flag that indicates the sources they consider unfit for use in cosmological analyses. The performance of each method was assessed through a set of informative metrics, using cross-matched spectroscopic and highlyaccurate photometric redshifts as the ground truth. We show that the rejection criteria set by participants are efficient in removing strong outliers, that is to say sources for which the photo-z deviates by more than 0.15(1 + z) from the spectroscopic-redshift (spec-z). We also show that, while all methods are able to provide reliable single value estimates, several machine-learning methods do not manage to produce useful PDZs. We find that no machine-learning method provides good results in the regions of galaxy color-space that are sparsely populated by spectroscopic-redshifts, for example z > 1. However they generally perform better than template-fitting methods at low redshift (z < 0.7), indicating that template-fitting methods do not use all of the information contained in the photometry. We introduce metrics that quantify both photo-z precision and completeness of the samples (post-rejection), since both contribute to the final figure of merit of the science goals of the survey (e.g., cosmic shear from Euclid). Template-fitting methods provide the best results in these metrics, but we show that a combination of template-fitting results and machine-learning results with rejection criteria can outperform any individual method. On this basis, we argue that further work in identifying how to best select between machine-learning and template-fitting approaches for each individual galaxy should be pursued as a priority

Euclid preparation. XXXI. The effect of the variations in photometric passbands on photometric-redshift accuracy

The technique of photometric redshifts has become essential for the exploitation of multi-band extragalactic surveys. While the requirements on photo-zs for the study of galaxy evolution mostly pertain to the precision and to the fraction of outliers, the most stringent requirement in their use in cosmology is on the accuracy, with a level of bias at the sub-percent level for the Euclid cosmology mission. A separate, and challenging, calibration process is needed to control the bias at this level of accuracy. The bias in photo-zs has several distinct origins that may not always be easily overcome. We identify here one source of bias linked to the spatial or time variability of the passbands used to determine the photometric colours of galaxies. We first quantified the effect as observed on several well-known photometric cameras, and found in particular that, due to the properties of optical filters, the redshifts of off-axis sources are usually overestimated. We show using simple simulations that the detailed and complex changes in the shape can be mostly ignored and that it is sufficient to know the mean wavelength of the passbands of each photometric observation to correct almost exactly for this bias; the key point is that this mean wavelength is independent of the spectral energy distribution of the source}. We use this property to propose a correction that can be computationally efficiently implemented in some photo-z algorithms, in particular template-fitting. We verified that our algorithm, implemented in the new photo-z code Phosphoros, can effectively reduce the bias in photo-zs on real data using the CFHTLS T007 survey, with an average measured bias Delta z over the redshift range 0.4<z<0.7 decreasing by about 0.02, specifically from Delta z~0.04 to Delta z~0.02 around z=0.5. Our algorithm is also able to produce corrected photometry for other applications.Comment: 19 pages, 13 figures; Accepted for publication in A&

Web enabled data management with DPM & LFC

The Disk Pool Manager (DPM) and LCG File Catalog (LFC) are two grid data management components currently used in production with more than 240 endpoints. Together with a set of grid client tools they give the users a unified view of their data, hiding most details concerning data location and access. Recently we’ve put a lot of effort in developing a reliable and high performance HTTP/WebDAV frontend to both our grid catalog and storage components, exposing the existing functionality to users accessing the services via standard clients - e.g. web browsers, curl - present in all operating systems, giving users a simple and straigh-forward way of interaction. In addition, as other relevant grid storage components (like dCache) expose their data using the same protocol, for the first time we had the opportunity of attempting a unified view of all grid storage using HTTP. We describe the mechanism used to integrate the grid catalog(s) with the multiple storage components - HTTP redirection -, including details on some assumptions made to allow integration with other implementations. We describe the way we hide the details regarding site availability or catalog inconsistencies, by switching the standard HTTP client automatically between multiple replicas. We also present measurements of access performance, and the relevant factors regarding replica selection - current throughput and load, geographic proximity, etc. Finally, we report on some additional work done to have this system as a viable alternative to GridFTP, providing multi-stream transfers and exploiting some additional features of WebDAV to enable third party copies - essential for managing data movements between storage systems - with equivalent performance

The DMLite Rucio Plugin: ATLAS data in a filesystem

Rucio is the next-generation data management system of the ATLAS experiment. Historically, clients interacted with the data management system via specialised tools, but in Rucio additional methods are provided. To support filesystem-like interaction with all ATLAS data, a plugin to the DMLite software stack has been developed. It is possible to mount Rucio as a filesystem, and execute regular filesystem operations in a POSIX fashion. This is exposed via various protocols, for example, WebDAV or NFS, which then removes any dependency on Rucio for client software. The main challenge for this work is the mapping of the set-like ATLAS namespace into a hierarchical filesystem, whilst preserving the high performance features of the former. This includes listing and searching for data, creation of files, datasets and containers, and the aggregation of existing data - all within directories with potentially millions of entries. This contribution details the design and implementation of the plugin. Furthermore, an evaluation of the performance characteristics is given, to show that this approach can scale to the requirements of ATLAS physics analysis