Two-term, asymptotically sharp estimates for eigenvalue means of the Laplacian

We present asymptotically sharp inequalities for the eigenvalues $\mu_k$ of the Laplacian on a domain with Neumann boundary conditions, using the averaged variational principle introduced in \cite{HaSt14}. For the Riesz mean $R_1(z)$ of the eigenvalues we improve the known sharp semiclassical bound in terms of the volume of the domain with a second term with the best possible expected power of $z$. In addition, we obtain two-sided bounds for individual $\mu_k$, which are semiclassically sharp. In a final section, we remark upon the Dirichlet case with the same methods

A nodal domain theorem and a higher-order Cheeger inequality for the graph pp-Laplacian

We consider the nonlinear graph $p$-Laplacian and its set of eigenvalues and associated eigenfunctions of this operator defined by a variational principle. We prove a nodal domain theorem for the graph $p$-Laplacian for any $p\geq 1$. While for $p>1$ the bounds on the number of weak and strong nodal domains are the same as for the linear graph Laplacian ($p=2$), the behavior changes for $p=1$. We show that the bounds are tight for $p\geq 1$ as the bounds are attained by the eigenfunctions of the graph $p$-Laplacian on two graphs. Finally, using the properties of the nodal domains, we prove a higher-order Cheeger inequality for the graph $p$-Laplacian for $p>1$. If the eigenfunction associated to the $k$-th variational eigenvalue of the graph $p$-Laplacian has exactly $k$ strong nodal domains, then the higher order Cheeger inequality becomes tight as $p\rightarrow 1$

On the optimal design of wall-to-wall heat transport

We consider the problem of optimizing heat transport through an incompressible fluid layer. Modeling passive scalar transport by advection-diffusion, we maximize the mean rate of total transport by a divergence-free velocity field. Subject to various boundary conditions and intensity constraints, we prove that the maximal rate of transport scales linearly in the r.m.s. kinetic energy and, up to possible logarithmic corrections, as the $1/3$rd power of the mean enstrophy in the advective regime. This makes rigorous a previous prediction on the near optimality of convection rolls for energy-constrained transport. Optimal designs for enstrophy-constrained transport are significantly more difficult to describe: we introduce a "branching" flow design with an unbounded number of degrees of freedom and prove it achieves nearly optimal transport. The main technical tool behind these results is a variational principle for evaluating the transport of candidate designs. The principle admits dual formulations for bounding transport from above and below. While the upper bound is closely related to the "background method", the lower bound reveals a connection between the optimal design problems considered herein and other apparently related model problems from mathematical materials science. These connections serve to motivate designs.Comment: Minor revisions from review. To appear in Comm. Pure Appl. Mat

Variational Quantum Fidelity Estimation

Computing quantum state fidelity will be important to verify and characterize states prepared on a quantum computer. In this work, we propose novel lower and upper bounds for the fidelity F(ρ,σ) based on the “truncated fidelity'” F(ρ_m,σ) which is evaluated for a state ρ_m obtained by projecting ρ onto its mm-largest eigenvalues. Our bounds can be refined, i.e., they tighten monotonically with mm. To compute our bounds, we introduce a hybrid quantum-classical algorithm, called Variational Quantum Fidelity Estimation, that involves three steps: (1) variationally diagonalize ρ, (2) compute matrix elements of σ in the eigenbasis of ρ, and (3) combine these matrix elements to compute our bounds. Our algorithm is aimed at the case where σ is arbitrary and ρ is low rank, which we call low-rank fidelity estimation, and we prove that no classical algorithm can efficiently solve this problem under reasonable assumptions. Finally, we demonstrate that our bounds can detect quantum phase transitions and are often tighter than previously known computable bounds for realistic situations

Approximate Inference with the Variational Holder Bound

We introduce the Variational Holder (VH) bound as an alternative to Variational Bayes (VB) for approximate Bayesian inference. Unlike VB which typically involves maximization of a non-convex lower bound with respect to the variational parameters, the VH bound involves minimization of a convex upper bound to the intractable integral with respect to the variational parameters. Minimization of the VH bound is a convex optimization problem; hence the VH method can be applied using off-the-shelf convex optimization algorithms and the approximation error of the VH bound can also be analyzed using tools from convex optimization literature. We present experiments on the task of integrating a truncated multivariate Gaussian distribution and compare our method to VB, EP and a state-of-the-art numerical integration method for this problem

Variational formulas and disorder regimes of random walks in random potentials

We give two variational formulas (qVar1) and (qVar2) for the quenched free energy of a random walk in random potential (RWRP) when (i) the underlying walk is directed or undirected, (ii) the environment is stationary and ergodic, and (iii) the potential is allowed to depend on the next step of the walk which covers random walk in random environment (RWRE). In the directed i.i.d. case, we also give two variational formulas (aVar1) and (aVar2) for the annealed free energy of RWRP. These four formulas are the same except that they involve infima over different sets, and the first two are modified versions of a previously known variational formula (qVar0) for which we provide a short alternative proof. Then, we show that (qVar0) always has a minimizer, (aVar2) never has any minimizers unless the RWRP is an RWRE, and (aVar1) has a minimizer if and only if the RWRP is in the weak disorder regime. In the latter case, the minimizer of (aVar1) is unique and it is also the unique minimizer of (qVar1), but (qVar2) has no minimizers except for RWRE. In the case of strong disorder, we give a sufficient condition for the nonexistence of minimizers of (qVar1) and (qVar2) which is satisfied for the log-gamma directed polymer with a sufficiently small parameter. We end with a conjecture which implies that (qVar1) and (qVar2) have no minimizers under very strong disorder.Comment: Published at http://dx.doi.org/10.3150/15-BEJ747 in the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm

On Variational Bounds of Mutual Information

Estimating and optimizing Mutual Information (MI) is core to many problems in machine learning; however, bounding MI in high dimensions is challenging. To establish tractable and scalable objectives, recent work has turned to variational bounds parameterized by neural networks, but the relationships and tradeoffs between these bounds remains unclear. In this work, we unify these recent developments in a single framework. We find that the existing variational lower bounds degrade when the MI is large, exhibiting either high bias or high variance. To address this problem, we introduce a continuum of lower bounds that encompasses previous bounds and flexibly trades off bias and variance. On high-dimensional, controlled problems, we empirically characterize the bias and variance of the bounds and their gradients and demonstrate the effectiveness of our new bounds for estimation and representation learning.Comment: ICML 201

Energy gaps of Hamiltonians from graph Laplacians

The Cheeger inequalities give an upper and lower bound on the spectral gap of discrete Laplacians defined on a graph in terms of the geometric characteristics of the graph. We generalise this approach and we employ it to determine if a given discrete Hamiltonian with non-positive elements is gapped or not in the thermodynamic limit. First, we define the graph that corresponds to such a generic Hamiltonian. Then we present a suitable generalisation of the Cheeger inequalities that overcomes scaling deficiencies of the original version. By employing simple examples we illustrate how the generalised Cheeger inequalities can successfully identify gapped or gapless phases and we comment on the computational complexity of this approach.Comment: 4+2 pages, 5 figures, including mathematical supplement, Ising model presentation clarifie

On some provably correct cases of variational inference for topic models

Variational inference is a very efficient and popular heuristic used in various forms in the context of latent variable models. It's closely related to Expectation Maximization (EM), and is applied when exact EM is computationally infeasible. Despite being immensely popular, current theoretical understanding of the effectiveness of variaitonal inference based algorithms is very limited. In this work we provide the first analysis of instances where variational inference algorithms converge to the global optimum, in the setting of topic models. More specifically, we show that variational inference provably learns the optimal parameters of a topic model under natural assumptions on the topic-word matrix and the topic priors. The properties that the topic word matrix must satisfy in our setting are related to the topic expansion assumption introduced in (Anandkumar et al., 2013), as well as the anchor words assumption in (Arora et al., 2012c). The assumptions on the topic priors are related to the well known Dirichlet prior, introduced to the area of topic modeling by (Blei et al., 2003). It is well known that initialization plays a crucial role in how well variational based algorithms perform in practice. The initializations that we use are fairly natural. One of them is similar to what is currently used in LDA-c, the most popular implementation of variational inference for topic models. The other one is an overlapping clustering algorithm, inspired by a work by (Arora et al., 2014) on dictionary learning, which is very simple and efficient. While our primary goal is to provide insights into when variational inference might work in practice, the multiplicative, rather than the additive nature of the variational inference updates forces us to use fairly non-standard proof arguments, which we believe will be of general interest.Comment: 46 pages, Compared to previous version: clarified notation, a number of typos fixed throughout pape

An approximation scheme for variational inequalities with convex and coercive Hamiltonians

We propose an approximation scheme for a class of semilinear variational inequalities whose Hamiltonian is convex and coercive. The proposed scheme is a natural extension of a previous splitting scheme proposed by Liang, Zariphopoulou and the author for semilinear parabolic PDEs. We establish the convergence of the scheme and determine the convergence rate by obtaining its error bounds. The bounds are obtained by Krylov's shaking coefficients technique and Barles-Jakobsen's optimal switching approximation, in which a key step is to introduce a variant switching system.Comment: arXiv admin note: text overlap with arXiv:1801.0058