An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval

Machine learning is now used in many areas of astrophysics, from detecting exoplanets in Kepler transit signals to removing telescope systematics. Recent work demonstrated the potential of using machine learning algorithms for atmospheric retrieval by implementing a random forest to perform retrievals in seconds that are consistent with the traditional, computationally-expensive nested-sampling retrieval method. We expand upon their approach by presenting a new machine learning model, \texttt{plan-net}, based on an ensemble of Bayesian neural networks that yields more accurate inferences than the random forest for the same data set of synthetic transmission spectra. We demonstrate that an ensemble provides greater accuracy and more robust uncertainties than a single model. In addition to being the first to use Bayesian neural networks for atmospheric retrieval, we also introduce a new loss function for Bayesian neural networks that learns correlations between the model outputs. Importantly, we show that designing machine learning models to explicitly incorporate domain-specific knowledge both improves performance and provides additional insight by inferring the covariance of the retrieved atmospheric parameters. We apply \texttt{plan-net} to the Hubble Space Telescope Wide Field Camera 3 transmission spectrum for WASP-12b and retrieve an isothermal temperature and water abundance consistent with the literature. We highlight that our method is flexible and can be expanded to higher-resolution spectra and a larger number of atmospheric parameters

Radiative Transfer and Inversion codes for characterizing planetary atmospheres: an overview

The study of planetary atmospheres is crucial for understanding the origin, evolution, and processes that shape celestial bodies like planets, moons and comets. The interpretation of planetary spectra requires a detailed understanding of radiative transfer (RT) and its application through computational codes. With the advancement of observations, atmospheric modelling, and inference techniques, diverse RT and retrieval codes in planetary science have been proliferated. However, the selection of the most suitable code for a given problem can be challenging. To address this issue, we present a comprehensive mini-overview of the different RT and retrieval codes currently developed or available in the field of planetary atmospheres. This study serves as a valuable resource for the planetary science community by providing a clear and accessible list of codes, and offers a useful reference for researchers and practitioners in their selection and application of RT and retrieval codes for planetary atmospheric studies.Comment: 10 pages, 1 figure, published in Frontiers in Astronomy and Space Sciences. https://www.frontiersin.org/articles/10.3389/fspas.2023.117674

Lessons learned from the 1st Ariel Machine Learning Challenge: Correcting transiting exoplanet light curves for stellar spots

The last decade has witnessed a rapid growth of the field of exoplanet discovery and characterisation. However, several big challenges remain, many of which could be addressed using machine learning methodology. For instance, the most prolific method for detecting exoplanets and inferring several of their characteristics, transit photometry, is very sensitive to the presence of stellar spots. The current practice in the literature is to identify the effects of spots visually and correct for them manually or discard the affected data. This paper explores a first step towards fully automating the efficient and precise derivation of transit depths from transit light curves in the presence of stellar spots. The primary focus of the paper is to present in detail a diverse arsenal of methods for doing so. The methods and results we present were obtained in the context of the 1st Machine Learning Challenge organized for the European Space Agency’s upcoming Ariel mission. We first present the problem, the simulated Ariel-like data and outline the Challenge while identifying best practices for organizing similar challenges in the future. Finally, we present the solutions obtained by the top-5 winning teams, provide their code and discuss their implications. Successful solutions either construct highly non-linear (w.r.t. the raw data) models with minimal preprocessing –deep neural networks and ensemble methods– or amount to obtaining meaningful statistics from the light curves, constructing linear models on which yields comparably good predictive performance

ESA-Ariel Data Challenge NeurIPS 2022: Inferring Physical Properties of Exoplanets From Next-Generation Telescopes

The study of extra-solar planets, or simply, exoplanets, planets outside our own Solar System, is fundamentally a grand quest to understand our place in the Universe. Discoveries in the last two decades have re-defined our understanding of planets, and helped us comprehend the uniqueness of our very own Earth. In recent years the focus has shifted from planet detection to planet characterisation, where key planetary properties are inferred from telescope observations using Monte Carlo-based methods. However, the efficiency of sampling-based methodologies is put under strain by the high-resolution observational data from next generation telescopes, such as the James Webb Space Telescope and the Ariel Space Mission. We are delighted to announce the acceptance of the Ariel ML Data Challenge 2022 as part of the NeurIPS competition track. The goal of this challenge is to identify a reliable and scalable method to perform planetary characterisation. Depending on the chosen track, participants are tasked to provide either quartile estimates or the approximate distribution of key planetary properties. To this end, a synthetic spectroscopic dataset has been generated from the official simulators for the ESA Ariel Space Mission. The aims of the competition are three-fold. 1) To offer a challenging application for comparing and advancing conditional density estimation methods. 2) To provide a valuable contribution towards reliable and efficient analysis of spectroscopic data, enabling astronomers to build a better picture of planetary demographics, and 3) To promote the interaction between ML and exoplanetary science. The competition is open from 15th June and will run until early October, participants of all skill levels are more than welcomed

Machine learning as an ultra-fast alternative to Bayesian retrievals

1Kapteyn Astronomical Institute, University of Groningen, Groningen, The Netherlands 2SRON Netherlands Institute for Space Research, Utrecht, The Netherlands 3Centre for Exoplanet Science, University of Edinburgh, Edinburgh, UK Introduction: Inferring physical and chemical properties of an exoplanet's atmosphere from its transmission spectrum is computationally expensive. A multitude of forward models, sampled from a high dimensional parameter space, need to be compared to the observation. The preferred sampling method is currently Nested Sampling [7], in particular, the MultiNest implementation [2, 3]. It typically requires tens to hundreds of thousands of forward models to converge. Therefore, simpler forward models are usually favoured over longer computation times. A possible workaround is to use machine learning. A machine learning algorithm trained on a grid of forward models and parameter pairs can perform retrievals in seconds. This would make it possible to use complex models that take full advantage of future facilities e.g., JWST. Not only would retrievals of individual exoplanets become much faster, but it would also enable statistical studies of populations of exoplanets. It would also be a valuable tool for retrievability analyses, for example to assess the sensitivity of using different chemical networks. The main obstacle to overcome is being able to predict accurate posterior distributions and error estimates on the retrieved parameters. These need to be as close as possible to their Bayesian counterparts. Methods: Expanding on the 5-parameter grid in [5], we used ARCiS (ARtful modelling Code for exoplanet Science) [6] to generate a grid of 200,000 forward models described by the following parameters: isothermal temperature (T), planetary radius (RP), planetary mass (MP), abundances of water (H2O), ammonia (NH3) and hydrogen cyanide (HCN), and cloud top pressure (Pcloud). The models contain 13 wavelength bins, matching those of WASP-12b's observation with HST/WFC3 [4]. We added normally distributed random noise with σ=50 ppm. We trained a random forest following the details in [5] and a convolutional neural network (CNN). We divided the data into a training set of 190,000 spectra and a test set of 10,000. For the CNN we reserved 19,000 spectra (10%) from the training set for validation. These are needed to update the network weights at each training iteration. The CNN was trained with the loss function introduced in [1] to output a probability distribution. To account for the observational noise, we combined the distributions predicted for multiple noisy copies of the spectrum. To evaluate the performance of the machine learning algorithms, we retrieved all the spectra in the test set and plotted our predictions against the true values for the parameters. We repeated the experiment with only 1,000 spectra for Nested Sampling, reflecting the increased computational overhead of each of these retrievals. We then used a transmission spectrum of WASP-12b observed with HST/WFC3 [4] as a real-world test case. Results: Although the random forest trains faster, the CNN provided better results. Figures 1 and 2 show the predicted versus the true parameters for the CNN and Nested Sampling bulk retrievals. Remarkably, we observe the same structures in both plots. This shows that the CNN is able to learn the relationship between spectral features and parameters. We also found that both the CNN and Nested Sampling provide correct error estimates, with ~60% of predictions within 1σ of the true value, ~98% within 2σ, and virtually all within 3σ. This is in almost perfect agreement with expectation from statistical errors. Figures 3 and 4 show the CNN and Nested Sampling retrieval of WASP-12b. Again, we see very similar results, although the CNN provides broader posterior distributions. Work is ongoing to try to fix this issue. We found that a training set of 180,000 spectra is unnecessarily large, and the same performance can be reached with only 20,000 spectra. This implies that the number of forward model computations needed to train a CNN is smaller than the number needed for a single Bayesian retrieval. If this holds for more complex forward models and higher quality spectra, it would make machine learning an extremely attractive alternative to Nested Sampling. Conclusion: The existing literature on machine learning retrievals of exoplanet atmospheres only has comparisons between machine learning and Nested Sampling for a handful of test cases [1, 5, 8]. In this work we present a comparison of bulk retrievals done with both methods, showing that machine learning can indeed be a viable and fast alternative to Nested Sampling. We are currently working on extending these results to models with equilibrium chemistry and to JWST/NIRSpec simulated spectra. Acknowledgements: This project has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 860470. References: [1] Cobb, A. D., Himes, M. D., Soboczenski, F., Zorzan, S., O'Beirne, M. D., Baydin, A. G., Gal, Y., Domagal-Goldman, S. D., Arney, G. N., Angerhausen, D. (2019, 5). An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval. [2] Feroz, F., Hobson, M. P. (2008). Monthly Notices of the Royal Astronomical Society 384 (2). [3] Feroz, F., Hobson, M. P., Bridges, M. (2009). Monthly Notices of the Royal Astronomical Society 398(4). [4] Kreidberg, L., Line, M. R., Bean, J. L., Stevenson, K. B., Desert, J.-M., Madhusudhan, N., Fortney, J. J., Barstow, J. K., Henry, G. W., Williamson, M. H., Showman, A. P. (2015). A DETECTION OF WATER IN THE TRANSMISSION SPECTRUM OF THE HOT JUPITER WASP-12b AND IMPLICATIONS FOR ITS ATMOSPHERIC COMPOSITION. Technical report. [5] Marquez-Neila, P., Fisher, C., Sznitman, R., Heng, K. (2018). Supervised machine learning for analysing spectra of exoplanetary atmospheres. [6] Min, M., Ormel, C. W., Chubb, K., Helling, C., Kawashima, Y. (2020). The ARCiS framework for exoplanet atmospheres. Astronomy & Astrophysics 642. [7] Skilling, J. (2006). Nested sampling for general Bayesian computation. Bayesian Analysis 1 (4). [8] Zingales, T., Waldmann, I. P. (2018). The Astronomical Journal 156 (6)

Expect the Unexpected: Deciphering Exoplanetary Signals with Machine Learning Techniques

The field of exoplanets has enjoyed unprecedented growth in the past decades, planets are being discovered at an exponential rate. With the launch of next-generation facilities in the coming decades, the arrival of high-quality spectroscopic data is expected to bring about yet another revolutionary change in our understanding of these remote worlds. The field has been actively developing tools to comprehend the large stream of incoming data, and among them, Machine Learning techniques are building up momentum as an alternative to conventional approaches. In this work, I developed methodologies to uncover potential biases in the interpretation of the exoplanetary atmosphere introduced during data analysis. I showed that naively combining observations from different instruments might lead to biased results, and in some extreme cases like WASP-96 b, it is impossible to com- bine observations. A new scheme of retrieval framework, namely the L - retrieval, holds the potential to detect incompatibility among different datasets by combining light-curve fitting with atmospheric radiative transfer modelling. This work also documents the application of ML techniques to two distinct fields of exoplanetary science: a planet signal detection pipeline for direct imaging data and a suite of diagnostic tools designed for the characterisation of exoplanets. In both approaches, I pioneered the integration of Explainable AI techniques to improve the reliability of the deep learning models. Initial successes of these novel methodologies have provided an exciting prospect to tackle upcoming challenges with the use of Artificial Intelligence. How- ever, significant work remains to progress these models from their current proof-of- concept stage to general application framework. In this thesis, I will discuss their current limitations, potential future, and the next steps required