Small-variance asymptotics for Bayesian neural networks

Bayesian neural networks (BNNs) are a rich and flexible class of models that have several advantages over standard feedforward networks, but are typically expensive to train on large-scale data. In this thesis, we explore the use of small-variance asymptotics-an approach to yielding fast algorithms from probabilistic models-on various Bayesian neural network models. We first demonstrate how small-variance asymptotics shows precise connections between standard neural networks and BNNs; for example, particular sampling algorithms for BNNs reduce to standard backpropagation in the small-variance limit. We then explore a more complex BNN where the number of hidden units is additionally treated as a random variable in the model. While standard sampling schemes would be too slow to be practical, our asymptotic approach yields a simple method for extending standard backpropagation to the case where the number of hidden units is not fixed. We show on several data sets that the resulting algorithm has benefits over backpropagation on networks with a fixed architecture.2019-01-02T00:00:00

On the exit time from a cone for random walks with drift

We compute the exponential decay of the probability that a given multi-dimensional random walk stays in a convex cone up to time $n$, as $n$ goes to infinity. We show that the latter equals the minimum, on the dual cone, of the Laplace transform of the random walk increments. As an example, our results find applications in the counting of walks in orthants, a classical domain in enumerative combinatorics.Comment: 21 pages, 2 figures, to appear in Revista Matem\'atica Iberoamerican

Orthogonal polynomial ensembles in probability theory

We survey a number of models from physics, statistical mechanics, probability theory and combinatorics, which are each described in terms of an orthogonal polynomial ensemble. The most prominent example is apparently the Hermite ensemble, the eigenvalue distribution of the Gaussian Unitary Ensemble (GUE), and other well-known ensembles known in random matrix theory like the Laguerre ensemble for the spectrum of Wishart matrices. In recent years, a number of further interesting models were found to lead to orthogonal polynomial ensembles, among which the corner growth model, directed last passage percolation, the PNG droplet, non-colliding random processes, the length of the longest increasing subsequence of a random permutation, and others. Much attention has been paid to universal classes of asymptotic behaviors of these models in the limit of large particle numbers, in particular the spacings between the particles and the fluctuation behavior of the largest particle. Computer simulations suggest that the connections go even farther and also comprise the zeros of the Riemann zeta function. The existing proofs require a substantial technical machinery and heavy tools from various parts of mathematics, in particular complex analysis, combinatorics and variational analysis. Particularly in the last decade, a number of fine results have been achieved, but it is obvious that a comprehensive and thorough understanding of the matter is still lacking. Hence, it seems an appropriate time to provide a surveying text on this research area.Comment: Published at http://dx.doi.org/10.1214/154957805100000177 in the Probability Surveys (http://www.i-journals.org/ps/) by the Institute of Mathematical Statistics (http://www.imstat.org