Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units

We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever and Hinton, 2008; Le Roux and Bengio, 2008, 2010; Mont\'ufar and Ay, 2011) to units with arbitrary finite state spaces, and the vanishing approximation error to an arbitrary approximation error tolerance. For example, we show that a $q$-ary deep belief network with $L\geq 2+\frac{q^{\lceil m-\delta \rceil}-1}{q-1}$ layers of width $n \leq m + \log_q(m) + 1$ for some $m\in \mathbb{N}$ can approximate any probability distribution on $\{0,1,\ldots,q-1\}^n$ without exceeding a Kullback-Leibler divergence of $\delta$. Our analysis covers discrete restricted Boltzmann machines and na\"ive Bayes models as special cases.Comment: 19 pages, 5 figures, 1 tabl

Evidence for a Lattice Weak Gravity Conjecture

The Weak Gravity Conjecture postulates the existence of superextremal charged particles, i.e. those with mass smaller than or equal to their charge in Planck units. We present further evidence for our recent observation that in known examples a much stronger statement is true: an infinite tower of superextremal particles of different charges exists. We show that effective Kaluza-Klein field theories and perturbative string vacua respect the Sublattice Weak Gravity Conjecture, namely that a finite index sublattice of the full charge lattice exists with a superextremal particle at each site. In perturbative string theory we show that this follows from modular invariance. However, we present counterexamples to the stronger possibility that a superextremal particle exists at every lattice site, including an example in which the lightest charged particle is subextremal. The Sublattice Weak Gravity Conjecture has many implications both for abstract theories of quantum gravity and for real-world physics. For instance, it implies that if a gauge group with very small coupling $e$ exists, then the fundamental gravitational cutoff energy of the theory is no higher than $\sim e^{1/3} M_{\rm Pl}$.Comment: v2: 41 pages, typos fixed, references added, substantial revisions and clarifications (conclusions unchanged

Robust regression with imprecise data

We consider the problem of regression analysis with imprecise data. By imprecise data we mean imprecise observations of precise quantities in the form of sets of values. In this paper, we explore a recently introduced likelihood-based approach to regression with such data. The approach is very general, since it covers all kinds of imprecise data (i.e. not only intervals) and it is not restricted to linear regression. Its result consists of a set of functions, reflecting the entire uncertainty of the regression problem. Here we study in particular a robust special case of the likelihood-based imprecise regression, which can be interpreted as a generalization of the method of least median of squares. Moreover, we apply it to data from a social survey, and compare it with other approaches to regression with imprecise data. It turns out that the likelihood-based approach is the most generally applicable one and is the only approach accounting for multiple sources of uncertainty at the same time

Hurewicz Theorem for Assouad-Nagata dimension

Given a function $f\colon X\to Y$ of metric spaces, its {\it asymptotic dimension} \asdim(f) is the supremum of \asdim(A) such that $A\subset X$ and \asdim(f(A))=0. Our main result is \begin{Thm} \label{ThmAInAbstract} \asdim(X)\leq \asdim(f)+\asdim(Y) for any large scale uniform function $f\colon X\to Y$. \end{Thm} \ref{ThmAInAbstract} generalizes a result of Bell and Dranishnikov in which $f$ is Lipschitz and $X$ is geodesic. We provide analogs of \ref{ThmAInAbstract} for Assouad-Nagata dimension $\dim_{AN}$ and asymptotic Assouad-Nagata dimension \ANasdim. In case of linearly controlled asymptotic dimension \Lasdim we provide counterexamples to three questions in a list of problems of Dranishnikov. As an application of analogs of \ref{ThmAInAbstract} we prove \begin{Thm} \label{ThmBInAbstract} If $1\to K\to G\to H\to 1$ is an exact sequence of groups and $G$ is finitely generated, then \ANasdim (G,d_G)\leq \ANasdim (K,d_G|K)+\ANasdim (H,d_H) for any word metrics metrics $d_G$ on $G$ and $d_H$ on $H$. \end{Thm} \ref{ThmBInAbstract} extends a result of Bell and Dranishnikov for asymptotic dimension

Power-law distributions in binned empirical data

Many man-made and natural phenomena, including the intensity of earthquakes, population of cities and size of international wars, are believed to follow power-law distributions. The accurate identification of power-law patterns has significant consequences for correctly understanding and modeling complex systems. However, statistical evidence for or against the power-law hypothesis is complicated by large fluctuations in the empirical distribution's tail, and these are worsened when information is lost from binning the data. We adapt the statistically principled framework for testing the power-law hypothesis, developed by Clauset, Shalizi and Newman, to the case of binned data. This approach includes maximum-likelihood fitting, a hypothesis test based on the Kolmogorov--Smirnov goodness-of-fit statistic and likelihood ratio tests for comparing against alternative explanations. We evaluate the effectiveness of these methods on synthetic binned data with known structure, quantify the loss of statistical power due to binning, and apply the methods to twelve real-world binned data sets with heavy-tailed patterns.Comment: Published in at http://dx.doi.org/10.1214/13-AOAS710 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Evaluating geometric queries using few arithmetic operations

Let \cp:=(P_1,...,P_s) be a given family of $n$-variate polynomials with integer coefficients and suppose that the degrees and logarithmic heights of these polynomials are bounded by $d$ and $h$, respectively. Suppose furthermore that for each $1\leq i\leq s$ the polynomial $P_i$ can be evaluated using $L$ arithmetic operations (additions, subtractions, multiplications and the constants 0 and 1). Assume that the family \cp is in a suitable sense \emph{generic}. We construct a database $\cal D$, supported by an algebraic computation tree, such that for each $x\in [0,1]^n$ the query for the signs of $P_1(x),...,P_s(x)$ can be answered using h d^{\cO(n^2)} comparisons and $nL$ arithmetic operations between real numbers. The arithmetic-geometric tools developed for the construction of $\cal D$ are then employed to exhibit example classes of systems of $n$ polynomial equations in $n$ unknowns whose consistency may be checked using only few arithmetic operations, admitting however an exponential number of comparisons

On the asymptotic geometry of abelian-by-cyclic groups

A finitely presented, torsion free, abelian-by-cyclic group can always be written as an ascending HNN extension Gamma_M of Z^n, determined by an n x n integer matrix M with det(M) \ne 0. The group Gamma_M is polycyclic if and only if |det(M)|=1. We give a complete classification of the nonpolycyclic groups Gamma_M up to quasi-isometry: given n x n integer matrices M,N with |det(M)|, |det(N)| > 1, the groups Gamma_M, Gamma_N are quasi-isometric if and only if there exist positive integers r,s such that M^r, N^s have the same absolute Jordan form. We also prove quasi-isometric rigidity: if Gamma_M is an abelian-by-cyclic group determined by an n x n integer matrix M with |det(M)| > 1, and if G is any finitely generated group quasi-isometric to Gamma_M, then there is a finite normal subgroup K of G such that G/K is abstractly commensurable to Gamma_N, for some n x n integer matrix N with |det(N)| > 1.Comment: 65 pages, 2 figures. To appear in Acta Mathematic