Small-variance asymptotics for Bayesian neural networks

Bayesian neural networks (BNNs) are a rich and flexible class of models that have several advantages over standard feedforward networks, but are typically expensive to train on large-scale data. In this thesis, we explore the use of small-variance asymptotics-an approach to yielding fast algorithms from probabilistic models-on various Bayesian neural network models. We first demonstrate how small-variance asymptotics shows precise connections between standard neural networks and BNNs; for example, particular sampling algorithms for BNNs reduce to standard backpropagation in the small-variance limit. We then explore a more complex BNN where the number of hidden units is additionally treated as a random variable in the model. While standard sampling schemes would be too slow to be practical, our asymptotic approach yields a simple method for extending standard backpropagation to the case where the number of hidden units is not fixed. We show on several data sets that the resulting algorithm has benefits over backpropagation on networks with a fixed architecture.2019-01-02T00:00:00

Can A Pseudo-Nambu-Goldstone Higgs Lead To Symmetry Non-Restoration?

The calculation of finite temperature contributions to the scalar potential in a quantum field theory is similar to the calculation of loop corrections at zero temperature. In natural extensions of the Standard Model where loop corrections to the Higgs potential cancel between Standard Model degrees of freedom and their symmetry partners, it is interesting to contemplate whether finite temperature corrections also cancel, raising the question of whether a broken phase of electroweak symmetry may persist at high temperature. It is well known that this does not happen in supersymmetric theories because the thermal contributions of bosons and fermions do not cancel each other. However, for theories with same spin partners, the answer is less obvious. Using the Twin Higgs model as a benchmark, we show that although thermal corrections do cancel at the level of quadratic divergences, subleading corrections still drive the system to a restored phase. We further argue that our conclusions generalize to other well-known extensions of the Standard Model where the Higgs is rendered natural by being the pseudo-Nambu-Goldstone mode of an approximate global symmetry.Comment: 21 pages, 5 figures. v2: fixed problem related to references with 9-digit arXiv identifiers. v3: references added v4: Some clarifications and more references added; matches published versio

Equivalence of Systematic Linear Data Structures and Matrix Rigidity

Recently, Dvir, Golovnev, and Weinstein have shown that sufficiently strong lower bounds for linear data structures would imply new bounds for rigid matrices. However, their result utilizes an algorithm that requires an $NP$ oracle, and hence, the rigid matrices are not explicit. In this work, we derive an equivalence between rigidity and the systematic linear model of data structures. For the $n$-dimensional inner product problem with $m$ queries, we prove that lower bounds on the query time imply rigidity lower bounds for the query set itself. In particular, an explicit lower bound of $\omega\left(\frac{n}{r}\log m\right)$ for $r$ redundant storage bits would yield better rigidity parameters than the best bounds due to Alon, Panigrahy, and Yekhanin. We also prove a converse result, showing that rigid matrices directly correspond to hard query sets for the systematic linear model. As an application, we prove that the set of vectors obtained from rank one binary matrices is rigid with parameters matching the known results for explicit sets. This implies that the vector-matrix-vector problem requires query time $\Omega(n^{3/2}/r)$ for redundancy $r \geq \sqrt{n}$ in the systematic linear model, improving a result of Chakraborty, Kamma, and Larsen. Finally, we prove a cell probe lower bound for the vector-matrix-vector problem in the high error regime, improving a result of Chattopadhyay, Kouck\'{y}, Loff, and Mukhopadhyay.Comment: 23 pages, 1 tabl