Polynomial Threshold Functions, AC^0 Functions and Spectral Norms

The class of polynomial-threshold functions is studied using harmonic analysis, and the results are used to derive lower bounds related to AC^0 functions. A Boolean function is polynomial threshold if it can be represented as a sign function of a sparse polynomial (one that consists of a polynomial number of terms). The main result is that polynomial-threshold functions can be characterized by means of their spectral representation. In particular, it is proved that a Boolean function whose L_1 spectral norm is bounded by a polynomial in n is a polynomial-threshold function, and that a Boolean function whose L_∞^(-1) spectral norm is not bounded by a polynomial in n is not a polynomial-threshold function. Some results for AC^0 functions are derived

Harmonic analysis of neural networks

Neural networks models have attracted a lot of interest in recent years mainly because there were perceived as a new idea for computing. These models can be described as a network in which every node computes a linear threshold function. One of the main difficulties in analyzing the properties of these networks is the fact that they consist of nonlinear elements. I will present a novel approach, based on harmonic analysis of Boolean functions, to analyze neural networks. In particular I will show how this technique can be applied to answer the following two fundamental questions (i) what is the computational power of a polynomial threshold element with respect to linear threshold elements? (ii) Is it possible to get exponentially many spurious memories when we use the outer-product method for programming the Hopfield model

Dynamic Bayesian smooth transition autoregressive models applied to hourly electricity load in southern Brazil

Dynamic Bayesian Smooth Transition Autoregressive (DBSTAR) models are proposed for nonlinear autoregressive time series processes as alternative to both the classical Smooth Transition Autoregressive (STAR) models of Chan and Tong (1986) and the Bayesian Simulation STAR (BSTAR) models of Lopes and Salazar (2005). Unlike those, DBSTAR models are sequential polynomial dynamic analytical models suitable for inherently non-stationary time series with non-linear characteristics such as asymmetric cycles. As they are analytical, they also avoid potential computational problems associated with BSTAR models and allow fast sequential estimation of parameters. Two types of DBSTAR models are defined here based on the method adopted to approximate the transition function of their autoregressive components, namely the Taylor and the B-splines DBSTAR models. A harmonic version of those models, that accounted for the cyclical component explicitly in a flexible yet parsimonious way, were applied to the well-known series of annual Canadian lynx trappings and showed improved fitting when compared to both the classical STAR and the BSTAR models. Another application to a long series of hourly electricity loading in southern Brazil, covering the period of the South-African Football World Cup in June 2010, illustrates the short-term forecasting accuracy of fast computing harmonic DBSTAR models that account for various characteristics such as periodic behaviour (both within-the-day and within-the-week) and average temperature

On the Power of Threshold Circuits with Small Weights

Linear threshold elements (LTEs) are the basic processing elements in artificial neural networks. An LTE computes a function that is a sign of a weighted sum of the input variables. The weights are arbitrary integers; actually they can be very big integers-exponential in the number of input variables. However, in practice, it is very difficult to implement big weights. So the natural question that one can ask is whether there is an efficient way to simulate a network of LTEs with big weights by a network of LTEs with small weights. We prove the following results: 1) every LTE with big weights can be simulated by a depth-3, polynomial size network of LTEs with small weights, 2) every depth-d polynomial size network of LTEs with big weights can be simulated by a depth-(2d+1), polynomial size network of LTEs with small weights. To prove these results, we use tools from harmonic analysis of Boolean functions. Our technique is quite general, it provides insights to some other problems. For example, we were able to improve the best known results on the depth of a network of threshold elements that computes the COMPARISON, ADDITION and PRODUCT of two n-bits numbers, and the MAXIMUM and the SORTING of n n-bit numbers

Invariance principle on the slice

We prove an invariance principle for functions on a slice of the Boolean cube, which is the set of all vectors {0,1}^n with Hamming weight k. Our invariance principle shows that a low-degree, low-influence function has similar distributions on the slice, on the entire Boolean cube, and on Gaussian space. Our proof relies on a combination of ideas from analysis and probability, algebra and combinatorics. Our result imply a version of majority is stablest for functions on the slice, a version of Bourgain's tail bound, and a version of the Kindler-Safra theorem. As a corollary of the Kindler-Safra theorem, we prove a stability result of Wilson's theorem for t-intersecting families of sets, improving on a result of Friedgut.Comment: 36 page

Introducing Mexican needlets for CMB analysis: Issues for practical applications and comparison with standard needlets

Over the last few years, needlets have a emerged as a useful tool for the analysis of Cosmic Microwave Background (CMB) data. Our aim in this paper is first to introduce in the CMB literature a different form of needlets, known as Mexican needlets, first discussed in the mathematical literature by Geller and Mayeli (2009a,b). We then proceed with an extensive study of the properties of both standard and Mexican needlets; these properties depend on some parameters which can be tuned in order to optimize the performance for a given application. Our second aim in this paper is then to give practical advice on how to adjust these parameters in order to achieve the best properties for a given problem in CMB data analysis. In particular we investigate localization properties in real and harmonic spaces and propose a recipe on how to quantify the influence of galactic and point source masks on the needlet coefficients. We also show that for certain parameter values, the Mexican needlets provide a close approximation to the Spherical Mexican Hat Wavelets (whence their name), with some advantages concerning their numerical implementation and the derivation of their statistical properties.Comment: 40 pages, 11 figures, published version, main modification: added section on more realistic galactic and point source mask