Bounding errors of Expectation-Propagation

Expectation Propagation is a very popular algorithm for variational inference, but comes with few theoretical guarantees. In this article, we prove that the approximation errors made by EP can be bounded. Our bounds have an asymptotic interpretation in the number $n$ of datapoints, which allows us to study EP's convergence with respect to the true posterior. In particular, we show that EP converges at a rate of $\mathcal{0}(n^{-2})$ for the mean, up to an order of magnitude faster than the traditional Gaussian approximation at the mode. We also give similar asymptotic expansions for moments of order 2 to 4, as well as excess Kullback-Leibler cost (defined as the additional KL cost incurred by using EP rather than the ideal Gaussian approximation). All these expansions highlight the superior convergence properties of EP. Our approach for deriving those results is likely applicable to many similar approximate inference methods. In addition, we introduce bounds on the moments of log-concave distributions that may be of independent interest.Comment: Accepted and published at NIPS 201

The OS* Algorithm: a Joint Approach to Exact Optimization and Sampling

Most current sampling algorithms for high-dimensional distributions are based on MCMC techniques and are approximate in the sense that they are valid only asymptotically. Rejection sampling, on the other hand, produces valid samples, but is unrealistically slow in high-dimension spaces. The OS* algorithm that we propose is a unified approach to exact optimization and sampling, based on incremental refinements of a functional upper bound, which combines ideas of adaptive rejection sampling and of A* optimization search. We show that the choice of the refinement can be done in a way that ensures tractability in high-dimension spaces, and we present first experiments in two different settings: inference in high-order HMMs and in large discrete graphical models.Comment: 21 page

Fourth Moment Theorems for Markov Diffusion Generators

Inspired by the insightful article arXiv:1210.7587, we revisit the Nualart-Peccati-criterion arXiv:math/0503598 (now known as the Fourth Moment Theorem) from the point of view of spectral theory of general Markov diffusion generators. We are not only able to drastically simplify all of its previous proofs, but also to provide new settings of diffusive generators (Laguerre, Jacobi) where such a criterion holds. Convergence towards gamma and beta distributions under moment conditions is also discussed.Comment: 15 page

Endogeneity and Instrumental Variables in Dynamic Models

The objective of the paper is to draw the theory of endogeneity in dynamic models in discrete and continuous time, in particular for diffusions and counting processes. We first provide an extension of the separable set-up to a separable dynamic framework given in term of semi-martingale decomposition. Then we define our function of interest as a stopping time for an additional noise process, whose role is played by a Brownian motion for diffusions, and a Poisson process for counting processes.

On the Equivalence between Herding and Conditional Gradient Algorithms

We show that the herding procedure of Welling (2009) takes exactly the form of a standard convex optimization algorithm--namely a conditional gradient algorithm minimizing a quadratic moment discrepancy. This link enables us to invoke convergence results from convex optimization and to consider faster alternatives for the task of approximating integrals in a reproducing kernel Hilbert space. We study the behavior of the different variants through numerical simulations. The experiments indicate that while we can improve over herding on the task of approximating integrals, the original herding algorithm tends to approach more often the maximum entropy distribution, shedding more light on the learning bias behind herding

Phase diagram of hard-core bosons on clean and disordered 2-leg ladders: Mott insulator - Luttinger liquid - Bose glass

One dimensional free-fermions and hard-core bosons are often considered to be equivalent. Indeed, when restricted to nearest-neighbor hopping on a chain the particles cannot exchange themselves, and therefore hardly experience their own statistics. Apart from the off-diagonal correlations which depends on the so-called Jordan-Wigner string, real-space observables are similar for free-fermions and hard-core bosons on a chain. Interestingly, by coupling only two chains, thus forming a two-leg ladder, particle exchange becomes allowed, and leads to a totally different physics between free-fermions and hard-core bosons. Using a combination of analytical (strong coupling, field theory, renormalization group) and numerical (quantum Monte Carlo, density-matrix renormalization group) approaches, we study the apparently simple but non-trivial model of hard-core bosons hopping in a two-leg ladder geometry. At half-filling, while a band insulator appears for fermions at large interchain hopping tperp >2t only, a Mott gap opens up for bosons as soon as tperp\neq0 through a Kosterlitz-Thouless transition. Away from half-filling, the situation is even more interesting since a gapless Luttinger liquid mode emerges in the symmetric sector with a non-trivial filling-dependent Luttinger parameter 1/2\leq Ks \leq 1. Consequences for experiments in cold atoms, spin ladders in a magnetic field, as well as disorder effects are discussed. In particular, a quantum phase transition is expected at finite disorder strength between a 1D superfluid and an insulating Bose glass phase.Comment: 24 pages, 23 figure

Genetically modified food and international trade: The case of India, Bangladesh, Indonesia, and the Philippines

"Genetically modified (GM) food crops have the potential to raise agricultural productivity in Asian countries, but they are also associated with the risk of market access losses in sensitive importing countries. We study the potential effects of introducing GM food crops in Bangladesh, India, Indonesia, and the Philippines in the presence of trade-related regulations of GM food in major importers. We focus on GM field crops (rice, wheat, maize, soybeans, and cotton) resistant to biotic and abiotic stresses, such as drought-resistant rice, and use a multi-country, multi-sector computable general equilibrium model. We build on previous international simulation models by improving the representation of the productivity shocks associated with GM crops, and by using an improved representation of the world market, accounting for the effects of GM food labeling policies in major importers and the possibility of segregation for non-GM products going toward sensitive importing countries. The results of our simulations first show that the gains associated with the adoption of GM food crops largely exceed any type of potential trade losses these countries may incur. Adopting GM crops also allows net importing countries to greatly reduce their imports. Overall, we find that GM rice is bound to be the most advantageous crop for the four countries. Second, we find that segregation of non-GM crops can help reduce any potential trade loss for GM adopters, such as India, that want to keep export opportunities in sensitive countries, even with a 5 percent segregation cost. Lastly, we find that the opportunity cost of segregation is much larger for sensitive importing countries than for countries adopting new GM crops, which suggests that sensitive importers will have the incentive to invest in separate non-GM marketing channels if exporting countries like India decide to adopt GM food crops." from Authors' AbstractGenetically modified food, International trade, Developing countries, Segregation,

Genetically Modified Rice, International Trade, and First-Mover Advantage: The Case of India and China

Crop Production/Industries, International Relations/Trade,

Nonparametric Analysis of Hedge Funds Lifetimes

Most of hedge funds databases are now keeping history of dead funds in order to control biases in empirical analysis. It is then possible to use these data for the analysis of hedge funds lifetimes and survivorship. This paper proposes two nonparametric specifications of duration models. First, the single risk model is an alternative to parametric duration models used in the literature. Second, the competing risks model consider the two reasons why hedge funds stop reporting. We apply the two models to hedge funds data and compare our results to the literature. In particular, we show that a cohort effect must be considered. Moreover, the reason of the exit is a crucial information for the analysis of funds' survival as for a large part of disappearing funds, exit cannot be explained by low performance or low level of assets.