A trust-region method for stochastic variational inference with applications to streaming data

Stochastic variational inference allows for fast posterior inference in complex Bayesian models. However, the algorithm is prone to local optima which can make the quality of the posterior approximation sensitive to the choice of hyperparameters and initialization. We address this problem by replacing the natural gradient step of stochastic varitional inference with a trust-region update. We show that this leads to generally better results and reduced sensitivity to hyperparameters. We also describe a new strategy for variational inference on streaming data and show that here our trust-region method is crucial for getting good performance.Comment: in Proceedings of the 32nd International Conference on Machine Learning, 201

The threshold for integer homology in random d-complexes

Let Y ~ Y_d(n,p) denote the Bernoulli random d-dimensional simplicial complex. We answer a question of Linial and Meshulam from 2003, showing that the threshold for vanishing of homology H_{d-1}(Y; Z) is less than 80d log n / n. This bound is tight, up to a constant factor.Comment: 12 pages, updated to include an additional torsion group boun

Recurrence and transience for the frog model on trees

The frog model is a growing system of random walks where a particle is added whenever a new site is visited. A longstanding open question is how often the root is visited on the infinite $d$-ary tree. We prove the model undergoes a phase transition, finding it recurrent for $d=2$ and transient for $d\geq 5$. Simulations suggest strong recurrence for $d=2$, weak recurrence for $d=3$, and transience for $d\geq 4$. Additionally, we prove a 0-1 law for all $d$-ary trees, and we exhibit a graph on which a 0-1 law does not hold. To prove recurrence when $d=2$, we construct a recursive distributional equation for the number of visits to the root in a smaller process and show the unique solution must be infinity a.s. The proof of transience when $d=5$ relies on computer calculations for the transition probabilities of a large Markov chain. We also include the proof for $d \geq 6$, which uses similar techniques but does not require computer assistance.Comment: 24 pages, 8 figures to appear in Annals of Probabilit