Non-convex Global Minimization and False Discovery Rate Control for the TREX

The TREX is a recently introduced method for performing sparse high-dimensional regression. Despite its statistical promise as an alternative to the lasso, square-root lasso, and scaled lasso, the TREX is computationally challenging in that it requires solving a non-convex optimization problem. This paper shows a remarkable result: despite the non-convexity of the TREX problem, there exists a polynomial-time algorithm that is guaranteed to find the global minimum. This result adds the TREX to a very short list of non-convex optimization problems that can be globally optimized (principal components analysis being a famous example). After deriving and developing this new approach, we demonstrate that (i) the ability of the preexisting TREX heuristic to reach the global minimum is strongly dependent on the difficulty of the underlying statistical problem, (ii) the new polynomial-time algorithm for TREX permits a novel variable ranking and selection scheme, (iii) this scheme can be incorporated into a rule that controls the false discovery rate (FDR) of included features in the model. To achieve this last aim, we provide an extension of the results of Barber & Candes (2015) to establish that the knockoff filter framework can be applied to the TREX. This investigation thus provides both a rare case study of a heuristic for non-convex optimization and a novel way of exploiting non-convexity for statistical inference

A Survey of Tuning Parameter Selection for High-dimensional Regression

Penalized (or regularized) regression, as represented by Lasso and its variants, has become a standard technique for analyzing high-dimensional data when the number of variables substantially exceeds the sample size. The performance of penalized regression relies crucially on the choice of the tuning parameter, which determines the amount of regularization and hence the sparsity level of the fitted model. The optimal choice of tuning parameter depends on both the structure of the design matrix and the unknown random error distribution (variance, tail behavior, etc). This article reviews the current literature of tuning parameter selection for high-dimensional regression from both theoretical and practical perspectives. We discuss various strategies that choose the tuning parameter to achieve prediction accuracy or support recovery. We also review several recently proposed methods for tuning-free high-dimensional regression.Comment: 28 pages, 2 figure

A prototype knockoff filter for group selection with FDR control

In many applications, we need to study a linear regression model that consists of a response variable and a large number of potential explanatory variables, and determine which variables are truly associated with the response. In Foygel Barber & Candès (2015, Ann. Statist., 43, 2055–2085), the authors introduced a new variable selection procedure called the knockoff filter to control the false discovery rate (FDR) and proved that this method achieves exact FDR control. In this paper, we propose a prototype knockoff filter for group selection by extending the Reid–Tibshirani (2016, Biostatistics, 17, 364–376) prototype method. Our prototype knockoff filter improves the computational efficiency and statistical power of the Reid–Tibshirani prototype method when it is applied for group selection. In some cases when the group features are spanned by one or a few hidden factors, we demonstrate that the Principal Component Analysis (PCA) prototype knockoff filter outperforms the Dai–Foygel Barber (2016, 33rd International Conference on Machine Learning (ICML 2016)) group knockoff filter. We present several numerical experiments to compare our prototype knockoff filter with the Reid–Tibshirani prototype method and the group knockoff filter. We have also conducted some analysis of the knockoff filter. Our analysis reveals that some knockoff path method statistics, including the Lasso path statistic, may lead to loss of power for certain design matrices and a specially designed response even if their signal strengths are still relatively strong

Alfvén waves underlying ionospheric destabilization: ground-based observations

During geomagnetic storms, terawatts of power in the million mile-per-hour solar wind pierce the Earth’s magnetosphere. Geomagnetic storms and substorms create transverse magnetic waves known as Alfvén waves. In the auroral acceleration region, Alfvén waves accelerate electrons up to one-tenth the speed of light via wave-particle interactions. These inertial Alfvén wave (IAW) accelerated electrons are imbued with sub-100 meter structure perpendicular to geomagnetic field B. The IAW electric field parallel to B accelerates electrons up to about 10 keV along B. The IAW dispersion relation quantifies the precipitating electron striation observed with high-speed cameras as spatiotemporally dynamic fine structured aurora. A network of tightly synchronized tomographic auroral observatories using model based iterative reconstruction (MBIR) techniques were developed in this dissertation. The TRANSCAR electron penetration model creates a basis set of monoenergetic electron beam eigenprofiles of auroral volume emission rate for the given location and ionospheric conditions. Each eigenprofile consists of nearly 200 broadband line spectra modulated by atmospheric attenuation, bandstop filter and imager quantum efficiency. The L-BFGS-B minimization routine combined with sub-pixel registered electron multiplying CCD video stream at order 10 ms cadence yields estimates of electron differential number flux at the top of the ionosphere. Our automatic data curation algorithm reduces one terabyte/camera/day into accurate MBIR-processed estimates of IAW-driven electron precipitation microstructure. This computer vision structured auroral discrimination algorithm was developed using a multiscale dual-camera system observing a 175 km and 14 km swath of sky simultaneously. This collective behavior algorithm exploits the “swarm” behavior of aurora, detectable even as video SNR approaches zero. A modified version of the algorithm is applied to topside ionospheric radar at Mars and broadcast FM passive radar. The fusion of data from coherent radar backscatter and optical data at order 10 ms cadence confirms and further quantifies the relation of strong Langmuir turbulence and streaming plasma upflows in the ionosphere with the finest spatiotemporal auroral dynamics associated with IAW acceleration. The software programs developed in this dissertation solve the century-old problem of automatically discriminating finely structured aurora from other forms and pushes the observational wave-particle science frontiers forward

Preemptively Pruning Clever-Hans Strategies in Deep Neural Networks

Robustness has become an important consideration in deep learning. With the help of explainable AI, mismatches between an explained model's decision strategy and the user's domain knowledge (e.g. Clever Hans effects) have been identified as a starting point for improving faulty models. However, it is less clear what to do when the user and the explanation agree. In this paper, we demonstrate that acceptance of explanations by the user is not a guarantee for a machine learning model to be robust against Clever Hans effects, which may remain undetected. Such hidden flaws of the model can nevertheless be mitigated, and we demonstrate this by contributing a new method, Explanation-Guided Exposure Minimization (EGEM), that preemptively prunes variations in the ML model that have not been the subject of positive explanation feedback. Experiments demonstrate that our approach leads to models that strongly reduce their reliance on hidden Clever Hans strategies, and consequently achieve higher accuracy on new data.Comment: 18 pages + supplemen