Sampling Methods for Unsupervised Learning

We present an algorithm to overcome the local maxima problem in estimating the parameters of mixture models. It combines existing approaches from both EM and a robust fitting algorithm, RANSAC, to give a data-driven stochastic learning scheme. Minimal subsets of data points, sufficient to constrain the parameters of the model, are drawn from proposal densities to discover new regions of high likelihood. The proposal densities are learnt using EM and bias the sampling toward promising solutions. The algorithm is computationally efficient, as well as effective at escaping from local maxima. We compare it with alternative methods, including EM and RANSAC, on both challenging synthetic data and the computer vision problem of alpha-matting

Coupling methods for multistage sampling

Multistage sampling is commonly used for household surveys when there exists no sampling frame, or when the population is scattered over a wide area. Multistage sampling usually introduces a complex dependence in the selection of the final units, which makes asymptotic results quite difficult to prove. In this work, we consider multistage sampling with simple random without replacement sampling at the first stage, and with an arbitrary sampling design for further stages. We consider coupling methods to link this sampling design to sampling designs where the primary sampling units are selected independently. We first generalize a method introduced by [Magyar Tud. Akad. Mat. Kutat\'{o} Int. K\"{o}zl. 5 (1960) 361-374] to get a coupling with multistage sampling and Bernoulli sampling at the first stage, which leads to a central limit theorem for the Horvitz--Thompson estimator. We then introduce a new coupling method with multistage sampling and simple random with replacement sampling at the first stage. When the first-stage sampling fraction tends to zero, this method is used to prove consistency of a with-replacement bootstrap for simple random without replacement sampling at the first stage, and consistency of bootstrap variance estimators for smooth functions of totals.Comment: Published at http://dx.doi.org/10.1214/15-AOS1348 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Elaborating Transition Interface Sampling Methods

We review two recently developed efficient methods for calculating rate constants of processes dominated by rare events in high-dimensional complex systems. The first is transition interface sampling (TIS), based on the measurement of effective fluxes through hypersurfaces in phase space. TIS improves efficiency with respect to standard transition path sampling (TPS) rate constant techniques, because it allows a variable path length and is less sensitive to recrossings. The second method is the partial path version of TIS. Developed for diffusive processes, it exploits the loss of long time correlation. We discuss the relation between the new techniques and the standard reactive flux methods in detail. Path sampling algorithms can suffer from ergodicity problems, and we introduce several new techniques to alleviate these problems, notably path swapping, stochastic configurational bias Monte Carlo shooting moves and order-parameter free path sampling. In addition, we give algorithms to calculate other interesting properties from path ensembles besides rate constants, such as activation energies and reaction mechanisms.Comment: 36 pages, 5 figure

Alignment methods for biased multicanonical sampling

The efficiency of the multicanonical procedure can be significantly improved by applying an additional bias to the numerically generated sample space. However, results obtained by biasing in different sampling regions cannot in general be accurately combined, since their relative normalization coefficient is not known precisely. We demonstrate that for overlapping biasing regions a simple iterative procedure can be employed to determine the required coefficients