Surprises in High-Dimensional Ridgeless Least Squares Interpolation

Interpolators -- estimators that achieve zero training error -- have attracted growing attention in machine learning, mainly because state-of-the art neural networks appear to be models of this type. In this paper, we study minimum $\ell_2$ norm (``ridgeless'') interpolation in high-dimensional least squares regression. We consider two different models for the feature distribution: a linear model, where the feature vectors $x_i \in {\mathbb R}^p$ are obtained by applying a linear transform to a vector of i.i.d.\ entries, $x_i = \Sigma^{1/2} z_i$ (with $z_i \in {\mathbb R}^p$); and a nonlinear model, where the feature vectors are obtained by passing the input through a random one-layer neural network, $x_i = \varphi(W z_i)$ (with $z_i \in {\mathbb R}^d$, $W \in {\mathbb R}^{p \times d}$ a matrix of i.i.d.\ entries, and $\varphi$ an activation function acting componentwise on $W z_i$). We recover -- in a precise quantitative way -- several phenomena that have been observed in large-scale neural networks and kernel machines, including the "double descent" behavior of the prediction risk, and the potential benefits of overparametrization.Comment: 68 pages; 16 figures. This revision contains non-asymptotic version of earlier results, and results for general coefficient

High-m Kink/Tearing Modes in Cylindrical Geometry

The global ideal kink equation, for cylindrical geometry and zero beta, is simplified in the high poloidal mode number limit and used to determine the tearing stability parameter, $\Delta^\prime$. In the presence of a steep monotonic current gradient, $\Delta^\prime$ becomes a function of a parameter, $\sigma_0$, characterising the ratio of the maximum current gradient to magnetic shear, and $x_s$, characterising the separation of the resonant surface from the maximum of the current gradient. In equilibria containing a current "spike", so that there is a non-monotonic current profile, $\Delta^\prime$ also depends on two parameters: $\kappa$, related to the ratio of the curvature of the current density at its maximum to the magnetic shear, and $x_s$, which now represents the separation of the resonance from the point of maximum current density. The relation of our results to earlier studies of tearing modes and to recent gyro-kinetic calculations of current driven instabilities, is discussed, together with potential implications for the stability of the tokamak pedestal.Comment: To appear in Plasma Physics and Controlled Fusio