Asymptotic Analysis of Machine Learning Models: Comparison Theorems and Universality

The study of Machine Learning models in asymptotic regimes, has provided insight into many of the properties of ML models, but seemingly contradicts classical statistical wisdom. To solve this mystery, this thesis focuses on the analysis of models such as the LASSO and Random features regression, when the data points and model parameters grow infinite at constant ratios. It provides analysis for the asymptotic behavior of these problems, including characterization of the learning curves; the predicted training and generalization error as a function of the degree of overparameterization. The papers in this thesis particularly focus on the usage of Gaussian comparison theorems as a methodological tool for the analysis of these problems. In particular, the convex Gaussian min max theorem allows us to study more complex ML optimization problems, by considering alternative models that are simpler to analyze, but asymptotically hold similar properties. Secondarily, this thesis considers universality, which within the asymptotic context demonstrates that many statistics of ML models are fully determined by lower order statistical moments. This allows us to study surrogate Gaussian models, matching these moments. These surrogate Gaussian models can subsequently be analyzed by means of the Gaussian comparison theorems

On the Distribution of Quadratic Expressions in Various Types of Random Vectors

Several approximations to the distribution of indefinite quadratic expressions in possibly singular Gaussian random vectors and ratios thereof are obtained in this dissertation. It is established that such quadratic expressions can be represented in their most general form as the difference of two positive definite quadratic forms plus a linear combination of Gaussian random variables. New advances on the distribution of quadratic expressions in elliptically contoured vectors, which are expressed as scalar mixtures of Gaussian vectors, are proposed as well. Certain distributional aspects of Hermitian quadratic expressions in complex Gaussian vectors are also investigated. Additionally, approximations to the distributions of quadratic forms in uniform, beta, exponential and gamma random variables as well as order statistics thereof are determined from their exact moments, for which explicit representations are derived. Closed form representations of the approximations to the density functions of the various types of quadratic expressions being considered herein are obtained by adjusting the base density functions associated with the quadratic forms appearing in the decompositions of the expressions by means of polynomials whose coefficients are determined from the moments of the target distributions. Quadratic forms being ubiquitous in Statistics, the proposed distributional results should prove eminently useful

On statistics of bi-orthogonal eigenvectors in real and complex Ginibre ensembles: combining partial Schur decomposition with supersymmetry

We suggest a method of studying the joint probability density (JPD) of an eigenvalue and the associated 'non-orthogonality overlap factor' (also known as the 'eigenvalue condition number') of the left and right eigenvectors for non-selfadjoint Gaussian random matrices of size $N\times N$. First we derive the general finite $N$ expression for the JPD of a real eigenvalue $\lambda$ and the associated non-orthogonality factor in the real Ginibre ensemble, and then analyze its 'bulk' and 'edge' scaling limits. The ensuing distribution is maximally heavy-tailed, so that all integer moments beyond normalization are divergent. A similar calculation for a complex eigenvalue $z$ and the associated non-orthogonality factor in the complex Ginibre ensemble is presented as well and yields a distribution with the finite first moment. Its 'bulk' scaling limit yields a distribution whose first moment reproduces the well-known result of Chalker and Mehlig \cite{ChalkerMehlig1998}, and we provide the 'edge' scaling distribution for this case as well. Our method involves evaluating the ensemble average of products and ratios of integer and half-integer powers of characteristic polynomials for Ginibre matrices, which we perform in the framework of a supersymmetry approach. Our paper complements recent studies by Bourgade and Dubach \cite{BourgadeDubach}.Comment: published versio

An Isocurvature CDM Cosmogony. II. Observational Tests

A companion paper presents a worked model for evolution through inflation to initial conditions for an isocurvature model for structure formation. It is shown here that the model is consistent with the available observational constraints that can be applied without the help of numerical simulations. The model gives an acceptable fit to the second moments of the angular fluctuations in the thermal background radiation and the second through fourth moments of the measured large-scale fluctuations in galaxy counts, within the possibly significant uncertainties in these measurements. The cluster mass function requires a rather low but observationally acceptable mass density, 0.1\lsim\Omega\lsim 0.2 in a cosmologically flat universe. Galaxies would be assembled earlier in this model than in the adiabatic version, an arguably good thing. Aspects of the predicted non-Gaussian character of the anisotropy of the thermal background radiation in this model are discussed.Comment: 14 pages, 3 postscript figures, uses aas2pp4.st

The Ellipticity of the Disks of Spiral Galaxies

The disks of spiral galaxies are generally elliptical rather than circular. The distribution of ellipticities can be fit with a log-normal distribution. For a sample of 12,764 galaxies from the Sloan Digital Sky Survey Data Release 1 (SDSS DR1), the distribution of apparent axis ratios in the i band is best fit by a log-normal distribution of intrinsic ellipticities with ln epsilon = -1.85 +/- 0.89. For a sample of nearly face-on spiral galaxies, analyzed by Andersen and Bershady using both photometric and spectroscopic data, the best fitting distribution of ellipticities has ln epsilon = -2.29 +/- 1.04. Given the small size of the Andersen-Bershady sample, the two distribution are not necessarily inconsistent. If the ellipticity of the potential were equal to that of the light distribution of the SDSS DR1 galaxies, it would produce 1.0 magnitudes of scatter in the Tully-Fisher relation, greater than is observed. The Andersen-Bershady results, however, are consistent with a scatter as small as 0.25 magnitudes in the Tully-Fisher relation.Comment: 19 pages, 5 figures; ApJ, accepte

Universal Results for Correlations of Characteristic Polynomials: Riemann-Hilbert Approach

We prove that general correlation functions of both ratios and products of characteristic polynomials of Hermitian random matrices are governed by integrable kernels of three different types: a) those constructed from orthogonal polynomials; b) constructed from Cauchy transforms of the same orthogonal polynomials and finally c) those constructed from both orthogonal polynomials and their Cauchy transforms. These kernels are related with the Riemann-Hilbert problem for orthogonal polynomials. For the correlation functions we obtain exact expressions in the form of determinants of these kernels. Derived representations enable us to study asymptotics of correlation functions of characteristic polynomials via Deift-Zhou steepest-descent/stationary phase method for Riemann-Hilbert problems, and in particular to find negative moments of characteristic polynomials. This reveals the universal parts of the correlation functions and moments of characteristic polynomials for arbitrary invariant ensemble of $\beta=2$ symmetry class.Comment: 34page

Tools for Dissecting Supernova Remnants Observed with Chandra: Methods and Application to the Galactic Remnant W49B

We introduce methods to quantify the X-ray morphologies of supernova remnants observed with the Chandra X-ray Telescope. These include a power-ratio technique to measure morphological asymmetries, correlation-length analysis to probe chemical segregation and distribution, and wavelet-transform analysis to quantify X-ray substructure. We demonstrate the utility and accuracy of these techniques on relevant synthetic data. Additionally, we show the methods' capabilities by applying them to the 55-ks Chandra ACIS observation of the galactic supernova remnant W49B. We analyze the images of prominent emission lines in W49B and use the results to discern physical properties. We find that the iron morphology is very distinct from the other elements: it is statistically more asymmetric, more segregated, and has 25% larger emitting substructures than the lighter ions. Comparatively, the silicon, sulfur, argon, and calcium are well-mixed, more isotropic, and have smaller, equally-sized emitting substructures. Based on fits of XMM-Newton spectra in regions identified as iron rich and iron poor, we determine that the iron in W49B must have been anisotropically ejected. We measure the abundance ratios in many regions, and we find that large, local variations are persistent throughout the remnant. We compare the mean, global abundance ratios to those predicted by spherical and bipolar core-collapse explosions; the results are consistent with a bipolar origin from a 25 solar mass progenitor. We calculate the filling factor of iron from the volume of its emitting substructures, enabling more precise mass estimates than previous studies. Overall, this work is a first step toward rigorously describing the physical properties of supernova remnants for comparison within and between sources.Comment: 51 pages, 24 figures, accepted by ApJ. For full resolution figures, see http://www.astro.ucsc.edu/~lopez/paper.html Fixed typo in URL; no other change