Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units

We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever and Hinton, 2008; Le Roux and Bengio, 2008, 2010; Mont\'ufar and Ay, 2011) to units with arbitrary finite state spaces, and the vanishing approximation error to an arbitrary approximation error tolerance. For example, we show that a $q$-ary deep belief network with $L\geq 2+\frac{q^{\lceil m-\delta \rceil}-1}{q-1}$ layers of width $n \leq m + \log_q(m) + 1$ for some $m\in \mathbb{N}$ can approximate any probability distribution on $\{0,1,\ldots,q-1\}^n$ without exceeding a Kullback-Leibler divergence of $\delta$. Our analysis covers discrete restricted Boltzmann machines and na\"ive Bayes models as special cases.Comment: 19 pages, 5 figures, 1 tabl

When Does a Mixture of Products Contain a Product of Mixtures?

We derive relations between theoretical properties of restricted Boltzmann machines (RBMs), popular machine learning models which form the building blocks of deep learning models, and several natural notions from discrete mathematics and convex geometry. We give implications and equivalences relating RBM-representable probability distributions, perfectly reconstructible inputs, Hamming modes, zonotopes and zonosets, point configurations in hyperplane arrangements, linear threshold codes, and multi-covering numbers of hypercubes. As a motivating application, we prove results on the relative representational power of mixtures of product distributions and products of mixtures of pairs of product distributions (RBMs) that formally justify widely held intuitions about distributed representations. In particular, we show that a mixture of products requiring an exponentially larger number of parameters is needed to represent the probability distributions which can be obtained as products of mixtures.Comment: 32 pages, 6 figures, 2 table

Modular localization and Wigner particles

We propose a framework for the free field construction of algebras of local observables which uses as an input the Bisognano-Wichmann relations and a representation of the Poincare' group on the one-particle Hilbert space. The abstract real Hilbert subspace version of the Tomita-Takesaki theory enables us to bypass some limitations of the Wigner formalism by introducing an intrinsic spacetime localization. Our approach works also for continuous spin representations to which we associate a net of von Neumann algebras on spacelike cones with the Reeh-Schlieder property. The positivity of the energy in the representation turns out to be equivalent to the isotony of the net, in the spirit of Borchers theorem. Our procedure extends to other spacetimes homogeneous under a group of geometric transformations as in the case of conformal symmetries and de Sitter spacetime.Comment: 22 pages, LaTeX. Some errors have been corrected. To appear on Rev. Math. Phy

Confidence intervals for partially identified parameters

In the last decade a growing body of research has studied inference on partially identified parameters (e.g., Manski, 1990, 2003). In many cases where the parameter of interest is realvalued, the identification region is an interval whose lower and upper bounds may be estimated from sample data. Confidence intervals may be constructed to take account of the sampling variation in estimates of these bounds. Horowitz and Manski (1998, 2000) proposed and applied interval estimates that asymptotically cover the entire identification region with fixed probability. Here we introduce conceptually different interval estimates that asymptotically cover each element in the identification region with fixed probability (but not necessarily every element simultaneously). We show that these two types of interval estimate are different in practice, the latter in general being shorter. The difference in length (in excess of the length of the identification set itself) can be substantial, and in large samples is comparable to the difference of one — and two—sided confidence intervals. A complication arises from the fact that the simplest version of the proposed interval is discontinuous in the limit case of point identification, leading to coverage rates that are not uniform in important subsets of the parameter space. We develop a modification depending on the width of the identification region that restores uniformity. We show that under some conditions, using the estimated width of the identification region instead of the true width maintains uniformity.

Globalization and Complementary Policies: Poverty Impacts in Rural Zambia

In this paper, we have two main objectives: to investigate the links between globalization and poverty observed in Zambia during the 1990s, and to explore the poverty impacts of non-traditional export growth. We look at consumption and income effects separately. On the consumption side, we study the maize marketing reforms and the elimination of maize subsidies. We find that complementary policies matter: the introduction of competition policies at the milling industry acted as a cushion that benefited consumers but the restriction on maize imports by small-scale mills hurt them. On the income side, we study agricultural export growth to estimate income gains from international trade. The gains are associated with market agriculture activities (such as growing cotton, tobacco, hybrid maize) and rural labor markets and wages. We find that by expanding trade opportunities Zambian households would earn significantly higher income. Securing these higher levels of well-being requires complementary policies, like the provision of infrastructure, credit, and extension services.

The WTO Doha Round, cotton sector dynamics, and poverty trends in Zambia

The Zambian cotton sector went through significant reforms during the 1990s. After a long period of parastatal control, a process of liberalization in cotton production and marketing began in 1994. These reforms were expected to benefit agricultural farmers. In Zambia, these are rural, often vulnerable, smallholders. The authors investigate the connection between the dynamics of the cotton sector and the dynamics of poverty and evaluate to what extent cotton can work as a vehicle for poverty alleviation. They find that cotton can indeed act as an effective mechanism for increased household welfare. They also find income gains associated with cotton production, as well as positive impacts on the long-run nutritional status of Zambian children. The impacts, however, are relatively small.Crops&Crop Management Systems,Environmental Economics&Policies,Agricultural Research,Economic Theory&Research,Livestock&Animal Husbandry

Regional Knowledge and the Emergence of an Industry: Laser Systems Production in West Germany, 1975-2005

We analyze the emergence and spatial evolution of the German laser systems industry. Regional knowledge in the related field of laser sources, as well as the presence of universities with physics or engineering departments, is conducive to the emergence of laser systems suppliers. The regional presence of source producers is also positively related to entry into laser systems. One important mechanism behind regional entry is the diversification of upstream laser source producers into the downstream systems market. Entry into the materials processing submarket appears to be unrelated to academic knowledge in the region, but the presence of laser source producers and the regional stock of laser knowledge are still highly predictive in this submarket.Innovation, regional knowledge, laser technology, emerging industries, diversification