182,642 research outputs found
An Ensemble of Bayesian Neural Networks for Exoplanetary Atmospheric Retrieval
Machine learning is now used in many areas of astrophysics, from detecting
exoplanets in Kepler transit signals to removing telescope systematics. Recent
work demonstrated the potential of using machine learning algorithms for
atmospheric retrieval by implementing a random forest to perform retrievals in
seconds that are consistent with the traditional, computationally-expensive
nested-sampling retrieval method. We expand upon their approach by presenting a
new machine learning model, \texttt{plan-net}, based on an ensemble of Bayesian
neural networks that yields more accurate inferences than the random forest for
the same data set of synthetic transmission spectra. We demonstrate that an
ensemble provides greater accuracy and more robust uncertainties than a single
model. In addition to being the first to use Bayesian neural networks for
atmospheric retrieval, we also introduce a new loss function for Bayesian
neural networks that learns correlations between the model outputs.
Importantly, we show that designing machine learning models to explicitly
incorporate domain-specific knowledge both improves performance and provides
additional insight by inferring the covariance of the retrieved atmospheric
parameters. We apply \texttt{plan-net} to the Hubble Space Telescope Wide Field
Camera 3 transmission spectrum for WASP-12b and retrieve an isothermal
temperature and water abundance consistent with the literature. We highlight
that our method is flexible and can be expanded to higher-resolution spectra
and a larger number of atmospheric parameters
Practical Bayesian Optimization of Machine Learning Algorithms
Machine learning algorithms frequently require careful tuning of model
hyperparameters, regularization terms, and optimization parameters.
Unfortunately, this tuning is often a "black art" that requires expert
experience, unwritten rules of thumb, or sometimes brute-force search. Much
more appealing is the idea of developing automatic approaches which can
optimize the performance of a given learning algorithm to the task at hand. In
this work, we consider the automatic tuning problem within the framework of
Bayesian optimization, in which a learning algorithm's generalization
performance is modeled as a sample from a Gaussian process (GP). The tractable
posterior distribution induced by the GP leads to efficient use of the
information gathered by previous experiments, enabling optimal choices about
what parameters to try next. Here we show how the effects of the Gaussian
process prior and the associated inference procedure can have a large impact on
the success or failure of Bayesian optimization. We show that thoughtful
choices can lead to results that exceed expert-level performance in tuning
machine learning algorithms. We also describe new algorithms that take into
account the variable cost (duration) of learning experiments and that can
leverage the presence of multiple cores for parallel experimentation. We show
that these proposed algorithms improve on previous automatic procedures and can
reach or surpass human expert-level optimization on a diverse set of
contemporary algorithms including latent Dirichlet allocation, structured SVMs
and convolutional neural networks
- …