A Kernel Perspective of Skip Connections in Convolutional Networks

Over-parameterized residual networks (ResNets) are amongst the most successful convolutional neural architectures for image processing. Here we study their properties through their Gaussian Process and Neural Tangent kernels. We derive explicit formulas for these kernels, analyze their spectra, and provide bounds on their implied condition numbers. Our results indicate that (1) with ReLU activation, the eigenvalues of these residual kernels decay polynomially at a similar rate compared to the same kernels when skip connections are not used, thus maintaining a similar frequency bias; (2) however, residual kernels are more locally biased. Our analysis further shows that the matrices obtained by these residual kernels yield favorable condition numbers at finite depths than those obtained without the skip connections, enabling therefore faster convergence of training with gradient descent

Medieval Emergencies and the Contemporary Debate

Abstract The contemporary debate on emergencies and the state of exception often relies on historical examples. Yet, the most recent discussions on the state of exception (a legal construct that deals with emergencies) also assume its modern inception. This article shows that medieval France formulated its own state of exception, meant to deal with emergencies, based on the legal principle of necessity. This article has two purposes. First, it challenges the historical narrative inherent in the contemporary debate, which assumes the modern inception of the state of exception. Second, it reinforces the trepidation with which many scholars today view the uses and abuses of the state of exception. This article does so by showing that the French crown used and abused the medieval principle of necessity in ways similar to current uses of the state of exception; it served similar purposes. Just as some scholars fear today, the French medieval state of exception often served as a pretext meant to change the legal order, turning the exception into the ordinary. The French crown used the state of exception to enhance its power, and it was central in the long process of building the early-modern French state