Spectrally-normalized margin bounds for neural networks

This paper presents a margin-based multiclass generalization bound for neural networks that scales with their margin-normalized "spectral complexity": their Lipschitz constant, meaning the product of the spectral norms of the weight matrices, times a certain correction factor. This bound is empirically investigated for a standard AlexNet network trained with SGD on the mnist and cifar10 datasets, with both original and random labels; the bound, the Lipschitz constants, and the excess risks are all in direct correlation, suggesting both that SGD selects predictors whose complexity scales with the difficulty of the learning task, and secondly that the presented bound is sensitive to this complexity.Comment: Comparison to arXiv v1: 1-norm in main bound refined to (2,1)-group-norm. Comparison to NIPS camera ready: typo fixe

Optimal Estimation and Prediction for Dense Signals in High-Dimensional Linear Models

Estimation and prediction problems for dense signals are often framed in terms of minimax problems over highly symmetric parameter spaces. In this paper, we study minimax problems over l2-balls for high-dimensional linear models with Gaussian predictors. We obtain sharp asymptotics for the minimax risk that are applicable in any asymptotic setting where the number of predictors diverges and prove that ridge regression is asymptotically minimax. Adaptive asymptotic minimax ridge estimators are also identified. Orthogonal invariance is heavily exploited throughout the paper and, beyond serving as a technical tool, provides additional insight into the problems considered here. Most of our results follow from an apparently novel analysis of an equivalent non-Gaussian sequence model with orthogonally invariant errors. As with many dense estimation and prediction problems, the minimax risk studied here has rate d/n, where d is the number of predictors and n is the number of observations; however, when d is roughly proportional to n the minimax risk is influenced by the spectral distribution of the predictors and is notably different from the linear minimax risk for the Gaussian sequence model (Pinsker, 1980) that often appears in other dense estimation and prediction problems.Comment: 29 pages, 0 figure

Prediction and Signal Extraction of Strong Dependent Processess in the Frequency Domain

We frequently observe that one of the aims of time series analysts is to predict future values of the data. For weakly dependent data, when the model is known up to a finite set of parameters, its statistical properties are well documented and exhaustively examined. However, if the model was misspecified, the predictors would no longer be correct. Motivated by this observation and due to the interest in obtaining adequate and reliable predictors, Bhansali (1974) examined the properties of a nonparametric predictor based on the canonical factorization of the spectral density function given in Whittle (1963) and known as FLES. However, the above work does not cover the so-called strongly dependent data. Due to the interest in this type of process, one of our objectives in this paper is to examine the properties of the FLES for these processes. In addition, we illustrate how the FLES can be adapted to recover the signal of a strongly dependent process, showing its consistency. The proposed method is semiparametric, in the sense that, in contrast to other methods, we do not need to assume any particular model for the noise except that it is weakly dependent.Prediction, strong dependence, spectral density function, canonical factorization, signal extraction.

Historical forest biomass dynamics modelled with Landsat spectral trajectories

Acknowledgements National Forest Inventory data are available online, provided by Ministerio de Agricultura, Alimentación y Medio Ambiente (España). Landsat images are available online, provided by the USGS.Peer reviewedPostprin

The Spitzer Atlas of Stellar Spectra

We present the Spitzer Atlas of Stellar Spectra (SASS), which includes 159 stellar spectra (5 to 32 mic; R~100) taken with the Infrared Spectrograph on the Spitzer Space Telescope. This Atlas gathers representative spectra of a broad section of the Hertzsprung-Russell diagram, intended to serve as a general stellar spectral reference in the mid-infrared. It includes stars from all luminosity classes, as well as Wolf-Rayet (WR) objects. Furthermore, it includes some objects of intrinsic interest, like blue stragglers and certain pulsating variables. All the spectra have been uniformly reduced, and all are available online. For dwarfs and giants, the spectra of early-type objects are relatively featureless, dominated by Hydrogen lines around A spectral types. Besides these, the most noticeable photospheric features correspond to water vapor and silicon monoxide in late-type objects and methane and ammonia features at the latest spectral types. Most supergiant spectra in the Atlas present evidence of circumstellar gas. The sample includes five M supergiant spectra, which show strong dust excesses and in some cases PAH features. Sequences of WR stars present the well-known pattern of lines of HeI and HeII, as well as forbidden lines of ionized metals. The characteristic flat-top shape of the [Ne III] line is evident even at these low spectral resolutions. Several Luminous Blue Variables and other transition stars are present in the Atlas and show very diverse spectra, dominated by circumstellar gas and dust features. We show that the [8]-[24] Spitzer colors (IRAC and MIPS) are poor predictors of spectral type for most luminosity classes.Comment: Accepted by ApJS; Atlas contents available from: http://web.ipac.caltech.edu/staff/ardila/Atlas/index.html; http://irsa.ipac.caltech.edu/data/SPITZER/SASS/; 70 PDF pages, including figure