16,899 research outputs found
An Affine-Invariant Sampler for Exoplanet Fitting and Discovery in Radial Velocity Data
Markov Chain Monte Carlo (MCMC) proves to be powerful for Bayesian inference
and in particular for exoplanet radial velocity fitting because MCMC provides
more statistical information and makes better use of data than common
approaches like chi-square fitting. However, the non-linear density functions
encountered in these problems can make MCMC time-consuming. In this paper, we
apply an ensemble sampler respecting affine invariance to orbital parameter
extraction from radial velocity data. This new sampler has only one free
parameter, and it does not require much tuning for good performance, which is
important for automatization. The autocorrelation time of this sampler is
approximately the same for all parameters and far smaller than
Metropolis-Hastings, which means it requires many fewer function calls to
produce the same number of independent samples. The affine-invariant sampler
speeds up MCMC by hundreds of times compared with Metropolis-Hastings in the
same computing situation. This novel sampler would be ideal for projects
involving large datasets such as statistical investigations of planet
distribution. The biggest obstacle to ensemble samplers is the existence of
multiple local optima; we present a clustering technique to deal with local
optima by clustering based on the likelihood of the walkers in the ensemble. We
demonstrate the effectiveness of the sampler on real radial velocity data.Comment: 24 pages, 7 figures, accepted to Ap
Fitness landscape of the cellular automata majority problem: View from the Olympus
In this paper we study cellular automata (CAs) that perform the computational
Majority task. This task is a good example of what the phenomenon of emergence
in complex systems is. We take an interest in the reasons that make this
particular fitness landscape a difficult one. The first goal is to study the
landscape as such, and thus it is ideally independent from the actual
heuristics used to search the space. However, a second goal is to understand
the features a good search technique for this particular problem space should
possess. We statistically quantify in various ways the degree of difficulty of
searching this landscape. Due to neutrality, investigations based on sampling
techniques on the whole landscape are difficult to conduct. So, we go exploring
the landscape from the top. Although it has been proved that no CA can perform
the task perfectly, several efficient CAs for this task have been found.
Exploiting similarities between these CAs and symmetries in the landscape, we
define the Olympus landscape which is regarded as the ''heavenly home'' of the
best local optima known (blok). Then we measure several properties of this
subspace. Although it is easier to find relevant CAs in this subspace than in
the overall landscape, there are structural reasons that prevent a searcher
from finding overfitted CAs in the Olympus. Finally, we study dynamics and
performance of genetic algorithms on the Olympus in order to confirm our
analysis and to find efficient CAs for the Majority problem with low
computational cost
Multiphase MCMC sampling for parameter inference in nonlinear ordinary differential equations
Traditionally, ODE parameter inference relies on solving the system of ODEs and assessing fit of the estimated signal with the observations. However, nonlinear ODEs often do not permit closed form solutions. Using numerical methods to solve the equations results in prohibitive computational costs, particularly when one adopts a Bayesian approach in sampling parameters from a posterior distribution. With the introduction of gradient matching, we can abandon the need to numerically solve the system of equations. Inherent in these efficient procedures is an introduction of bias to the learning problem as we no longer sample based on the exact likelihood function. This paper presents a multiphase MCMC approach that attempts to close the gap between efficiency and accuracy. By sampling using a surrogate likelihood, we accelerate convergence to the stationary distribution before sampling using the exact likelihood. We demonstrate that this method combines the efficiency of gradient matching and the accuracy of the exact likelihood scheme
Multimodal estimation of distribution algorithms
Taking the advantage of estimation of distribution algorithms (EDAs) in preserving high diversity, this paper proposes a multimodal EDA. Integrated with clustering strategies for crowding and speciation, two versions of this algorithm are developed, which operate at the niche level. Then these two algorithms are equipped with three distinctive techniques: 1) a dynamic cluster sizing strategy; 2) an alternative utilization of Gaussian and Cauchy distributions to generate offspring; and 3) an adaptive local search. The dynamic cluster sizing affords a potential balance between exploration and exploitation and reduces the sensitivity to the cluster size in the niching methods. Taking advantages of Gaussian and Cauchy distributions, we generate the offspring at the niche level through alternatively using these two distributions. Such utilization can also potentially offer a balance between exploration and exploitation. Further, solution accuracy is enhanced through a new local search scheme probabilistically conducted around seeds of niches with probabilities determined self-adaptively according to fitness values of these seeds. Extensive experiments conducted on 20 benchmark multimodal problems confirm that both algorithms can achieve competitive performance compared with several state-of-the-art multimodal algorithms, which is supported by nonparametric tests. Especially, the proposed algorithms are very promising for complex problems with many local optima
Visualising Basins of Attraction for the Cross-Entropy and the Squared Error Neural Network Loss Functions
Quantification of the stationary points and the associated basins of
attraction of neural network loss surfaces is an important step towards a
better understanding of neural network loss surfaces at large. This work
proposes a novel method to visualise basins of attraction together with the
associated stationary points via gradient-based random sampling. The proposed
technique is used to perform an empirical study of the loss surfaces generated
by two different error metrics: quadratic loss and entropic loss. The empirical
observations confirm the theoretical hypothesis regarding the nature of neural
network attraction basins. Entropic loss is shown to exhibit stronger gradients
and fewer stationary points than quadratic loss, indicating that entropic loss
has a more searchable landscape. Quadratic loss is shown to be more resilient
to overfitting than entropic loss. Both losses are shown to exhibit local
minima, but the number of local minima is shown to decrease with an increase in
dimensionality. Thus, the proposed visualisation technique successfully
captures the local minima properties exhibited by the neural network loss
surfaces, and can be used for the purpose of fitness landscape analysis of
neural networks.Comment: Preprint submitted to the Neural Networks journa
- …