What makes waveform-based deep learning so hard? Despite numerous attempts at
training convolutional neural networks (convnets) for filterbank design, they
often fail to outperform hand-crafted baselines. These baselines are linear
time-invariant systems: as such, they can be approximated by convnets with wide
receptive fields. Yet, in practice, gradient-based optimization leads to
suboptimal approximations. In our article, we approach this phenomenon from the
perspective of initialization. We present a theory of large deviations for the
energy response of FIR filterbanks with random Gaussian weights. We find that
deviations worsen for large filters and locally periodic input signals, which
are both typical for audio signal processing applications. Numerical
simulations align with our theory and suggest that the condition number of a
convolutional layer follows a logarithmic scaling law between the number and
length of the filters, which is reminiscent of discrete wavelet bases.Comment: 4 pages, 5 figures, 1 page appendix, under review for IEEE SP