Nonparametric Estimation of Multi-View Latent Variable Models

Spectral methods have greatly advanced the estimation of latent variable models, generating a sequence of novel and efficient algorithms with strong theoretical guarantees. However, current spectral algorithms are largely restricted to mixtures of discrete or Gaussian distributions. In this paper, we propose a kernel method for learning multi-view latent variable models, allowing each mixture component to be nonparametric. The key idea of the method is to embed the joint distribution of a multi-view latent variable into a reproducing kernel Hilbert space, and then the latent parameters are recovered using a robust tensor power method. We establish that the sample complexity for the proposed method is quadratic in the number of latent components and is a low order polynomial in the other relevant parameters. Thus, our non-parametric tensor approach to learning latent variable models enjoys good sample and computational efficiencies. Moreover, the non-parametric tensor power method compares favorably to EM algorithm and other existing spectral algorithms in our experiments

Three-Body Recombination near a Narrow Feshbach Resonance in 6 Li

We experimentally measure and theoretically analyze the three-atom recombination rate, L3, around a narrow s-wave magnetic Feshbach resonance of 6Li−6Li at 543.3 G. By examining both the magnetic field dependence and, especially, the temperature dependence of L3 over a wide range of temperatures from a few μK to above 200 μK, we show that three-atom recombination through a narrow resonance follows a universal behavior determined by the long-range van der Waals potential and can be described by a set of rate equations in which three-body recombination proceeds via successive pairwise interactions. We expect the underlying physical picture to be applicable not only to narrow s wave resonances, but also to resonances in nonzero partial waves, and not only at ultracold temperatures, but also at much higher temperatures

Scalable Kernel Methods via Doubly Stochastic Gradients

The general perception is that kernel methods are not scalable, and neural nets are the methods of choice for nonlinear learning problems. Or have we simply not tried hard enough for kernel methods? Here we propose an approach that scales up kernel methods using a novel concept called "doubly stochastic functional gradients". Our approach relies on the fact that many kernel methods can be expressed as convex optimization problems, and we solve the problems by making two unbiased stochastic approximations to the functional gradient, one using random training points and another using random functions associated with the kernel, and then descending using this noisy functional gradient. We show that a function produced by this procedure after $t$ iterations converges to the optimal function in the reproducing kernel Hilbert space in rate $O(1/t)$, and achieves a generalization performance of $O(1/\sqrt{t})$. This doubly stochasticity also allows us to avoid keeping the support vectors and to implement the algorithm in a small memory footprint, which is linear in number of iterations and independent of data dimension. Our approach can readily scale kernel methods up to the regimes which are dominated by neural nets. We show that our method can achieve competitive performance to neural nets in datasets such as 8 million handwritten digits from MNIST, 2.3 million energy materials from MolecularSpace, and 1 million photos from ImageNet.Comment: 32 pages, 22 figure

Comparison of two efficient methods for calculating partition functions

In the long-time pursuit of the solution to calculate the partition function (or free energy) of condensed matter, Monte-Carlo-based nested sampling should be the state-of-the-art method, and very recently, we established a direct integral approach that works at least four orders faster. In present work, the above two methods were applied to solid argon at temperatures up to $300$K, and the derived internal energy and pressure were compared with the molecular dynamics simulation as well as experimental measurements, showing that the calculation precision of our approach is about 10 times higher than that of the nested sampling method.Comment: 6 pages, 4 figure