Probability density estimation of photometric redshifts based on machine learning

Photometric redshifts (photo-z's) provide an alternative way to estimate the distances of large samples of galaxies and are therefore crucial to a large variety of cosmological problems. Among the various methods proposed over the years, supervised machine learning (ML) methods capable to interpolate the knowledge gained by means of spectroscopical data have proven to be very effective. METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts) is a novel method designed to provide a reliable PDF (Probability density Function) of the error distribution of photometric redshifts predicted by ML methods. The method is implemented as a modular workflow, whose internal engine for photo-z estimation makes use of the MLPQNA neural network (Multi Layer Perceptron with Quasi Newton learning rule), with the possibility to easily replace the specific machine learning model chosen to predict photo-z's. After a short description of the software, we present a summary of results on public galaxy data (Sloan Digital Sky Survey - Data Release 9) and a comparison with a completely different method based on Spectral Energy Distribution (SED) template fitting.Comment: 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016 784995

METAPHOR: a machine-learning-based method for the probability density estimation of photometric redshifts

A variety of fundamental astrophysical science topics require the determination of very accurate photometric redshifts (photo-z). A wide plethora of methods have been developed, based either on template models fitting or on empirical explorations of the photometric parameter space. Machine-learning-based techniques are not explicitly dependent on the physical priors and able to produce accurate photo-z estimations within the photometric ranges derived from the spectroscopic training set. These estimates, however, are not easy to characterize in terms of a photo-z probability density function (PDF), due to the fact that the analytical relation mapping the photometric parameters on to the redshift space is virtually unknown. We present METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts), a method designed to provide a reliable PDF of the error distribution for empirical techniques. The method is implemented as a modular workflow, whose internal engine for photo-z estimation makes use of the MLPQNA neural network (Multi Layer Perceptron with Quasi Newton learning rule), with the possibility to easily replace the specific machine-learning model chosen to predict photo-z. We present a summary of results on SDSS-DR9 galaxy data, used also to perform a direct comparison with PDFs obtained by the LE PHARE spectral energy distribution template fitting. We show that METAPHOR is capable to estimate the precision and reliability of photometric redshifts obtained with three different self-adaptive techniques, I.e. MLPQNA, Random Forest and the standard K-Nearest Neighbors models

XMMPZCAT: A catalogue of photometric redshifts for X-ray sources

The third version of the XMM-Newton serendipitous catalogue (3XMM), containing almost half million sources, is now the largest X-ray catalogue. However, its full scientific potential remains untapped due to the lack of distance information (i.e. redshifts) for the majority of its sources. Here we present XMMPZCAT, a catalogue of photometric redshifts (photo-z) for 3XMM sources. We searched for optical counterparts of 3XMM-DR6 sources outside the Galactic plane in the SDSS and Pan-STARRS surveys, with the addition of near- (NIR) and mid-infrared (MIR) data whenever possible (2MASS, UKIDSS, VISTA-VHS, and AllWISE). We used this photometry data set in combination with a training sample of 5157 X-ray selected sources and the MLZ-TPZ package, a supervised machine learning algorithm based on decision trees and random forests for the calculation of photo-z. We have estimated photo-z for 100,178 X-ray sources, about 50% of the total number of 3XMM sources (205,380) in the XMM-Newton fields selected to build this catalogue (4208 out of 9159). The accuracy of our results highly depends on the available photometric data, with a rate of outliers ranging from 4% for sources with data in the optical+NIR+MIR, up to $\sim$40% for sources with only optical data. We also addressed the reliability level of our results by studying the shape of the photo-z probability density distributions.Comment: 16 pages, 14 figures, A&A accepte

Photometric redshift estimation via deep learning

The need to analyze the available large synoptic multi-band surveys drives the development of new data-analysis methods. Photometric redshift estimation is one field of application where such new methods improved the results, substantially. Up to now, the vast majority of applied redshift estimation methods have utilized photometric features. We aim to develop a method to derive probabilistic photometric redshift directly from multi-band imaging data, rendering pre-classification of objects and feature extraction obsolete. A modified version of a deep convolutional network was combined with a mixture density network. The estimates are expressed as Gaussian mixture models representing the probability density functions (PDFs) in the redshift space. In addition to the traditional scores, the continuous ranked probability score (CRPS) and the probability integral transform (PIT) were applied as performance criteria. We have adopted a feature based random forest and a plain mixture density network to compare performances on experiments with data from SDSS (DR9). We show that the proposed method is able to predict redshift PDFs independently from the type of source, for example galaxies, quasars or stars. Thereby the prediction performance is better than both presented reference methods and is comparable to results from the literature. The presented method is extremely general and allows us to solve of any kind of probabilistic regression problems based on imaging data, for example estimating metallicity or star formation rate of galaxies. This kind of methodology is tremendously important for the next generation of surveys.Comment: 16 pages, 12 figures, 6 tables. Accepted for publication on A&

Data Deluge in Astrophysics: Photometric Redshifts as a Template Use Case

Astronomy has entered the big data era and Machine Learning based methods have found widespread use in a large variety of astronomical applications. This is demonstrated by the recent huge increase in the number of publications making use of this new approach. The usage of machine learning methods, however is still far from trivial and many problems still need to be solved. Using the evaluation of photometric redshifts as a case study, we outline the main problems and some ongoing efforts to solve them.Comment: 13 pages, 3 figures, Springer's Communications in Computer and Information Science (CCIS), Vol. 82

METAPHOR: Probability density estimation for machine learning based photometric redshifts

We present METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts), a method able to provide a reliable PDF for photometric galaxy redshifts estimated through empirical techniques. METAPHOR is a modular workflow, mainly based on the MLPQNA neural network as internal engine to derive photometric galaxy redshifts, but giving the possibility to easily replace MLPQNA with any other method to predict photo-z's and their PDF. We present here the results about a validation test of the workflow on the galaxies from SDSS-DR9, showing also the universality of the method by replacing MLPQNA with KNN and Random Forest models. The validation test include also a comparison with the PDF's derived from a traditional SED template fitting method (Le Phare).Comment: proceedings of the International Astronomical Union, IAU-325 symposium, Cambridge University pres

Estimating Photometric Redshifts for X-ray sources in the X-ATLAS field, using machine-learning techniques

We present photometric redshifts for 1,031 X-ray sources in the X-ATLAS field, using the machine learning technique TPZ (Carrasco Kind & Brunner 2013). X-ATLAS covers 7.1 deg2 observed with the XMM-Newton within the Science Demonstration Phase (SDP) of the H-ATLAS field, making it one of the largest contiguous areas of the sky with both XMMNewton and Herschel coverage. All of the sources have available SDSS photometry while 810 have additionally mid-IR and/or near-IR photometry. A spectroscopic sample of 5,157 sources primarily in the XMM/XXL field, but also from several X-ray surveys and the SDSS DR13 redshift catalogue, is used for the training of the algorithm. Our analysis reveals that the algorithm performs best when the sources are split, based on their optical morphology, into point-like and extended sources. Optical photometry alone is not enough for the estimation of accurate photometric redshifts, but the results greatly improve when, at least, mid-IR photometry is added in the training process. In particular, our measurements show that the estimated photometric redshifts for the X-ray sources of the training sample, have a normalized absolute median deviation, n_mad=0.06, and the percentage of outliers, eta=10-14 percent, depending on whether the sources are extended or point-like. Our final catalogue contains photometric redshifts for 933 out of the 1,031 X-ray sources with a median redshift of 0.9.Comment: 10 pages, 13 figures, A&A accepte