Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review

The paper characterizes classes of functions for which deep learning can be exponentially better than shallow learning. Deep convolutional networks are a special case of these conditions, though weight sharing is not the main reason for their exponential advantage

Numerical Analysis of Target Enumeration via Euler Characteristic Integrals: 2 Dimensional Disk Supports

In this paper, we look at the types of errors that are introduced when using the Euler characteristic integral approach to count the number of targets in a discrete sensor field. In contrast to other theoretical analyses, this paper examines discrete sensor fields rather than continuous ones. The probabilities of first and second order errors are worked out combinatorially, and a general formula is found which is discovered to be proportional to much higher order errors. Asymptotic results are also derived. Thus, this work gives us the bias of the Euler characteristic integral as a point estimator for the number of targets in a sensor field, and a general understanding of how the Euler characteristic integral fails, or more often succeeds.Comment: 20 pages, 12 figures, 2 table

Phase Harmonic Correlations and Convolutional Neural Networks

A major issue in harmonic analysis is to capture the phase dependence of frequency representations, which carries important signal properties. It seems that convolutional neural networks have found a way. Over time-series and images, convolutional networks often learn a first layer of filters which are well localized in the frequency domain, with different phases. We show that a rectifier then acts as a filter on the phase of the resulting coefficients. It computes signal descriptors which are local in space, frequency and phase. The non-linear phase filter becomes a multiplicative operator over phase harmonics computed with a Fourier transform along the phase. We prove that it defines a bi-Lipschitz and invertible representation. The correlations of phase harmonics coefficients characterise coherent structures from their phase dependence across frequencies. For wavelet filters, we show numerically that signals having sparse wavelet coefficients can be recovered from few phase harmonic correlations, which provide a compressive representationComment: 26 pages, 8 figure