A Regularized Interior-Point Method for Constrained Linear Least Squares

RÉSUMÉ : Nous proposons une méthode de points intérieurs non réalisable pour le problème aux moindres carrés linéaire avec contraintes basée sur la régularisation primale-duale de problèmes quadratiques convexes de Friedlander et Orban (2012). À chaque itération, la méthode effectue une factorisation LDLT creuse d’une matrice symétrique et quasi définie. Cette matrice est uniformément bornée et non singulière. Nous établissons des conditions sous lesquelles la méthode produit une solution du problème original. La régularisation nous permet d’éliminer l’hypothèse que les gradients actifs sont linéairement indépendants. Bien que l’implémentation proposée ici repose sur une factorisation, elle ouvre la voie à une implémentation itérative dans laquelle on résout un problème aux moindres carrés régularisé sans contraintes de façon inexacte à chaque itération. Nous illustrons notre approche sur plusieurs applications qui mettent en évidence ses avantages.----------ABSTRACT : We propose an infeasible interior-point algorithm for constrained linear least-squares problems based on the primal-dual regularization of convex programs of Friedlander and Orban (2012). At each iteration, the sparse LDLT factorization of a symmetric quasi-definite matrix is computed. This coefficient matrix is shown to be uniformly bounded and nonsingular. We establish conditions under which a solution of the original problem is recovered. The regularization allows us to dispense with the assumption that the active gradients are linearly independent. Although the implementation described here is factorization based, it paves the way for a matrix-free implementation in which a regularized unconstrained linear least-squares problem is solved at each iteration. We report on computational experience and illustrate the potential advantages of our approach

A Generic Path Algorithm for Regularized Statistical Estimation

Regularization is widely used in statistics and machine learning to prevent overfitting and gear solution towards prior information. In general, a regularized estimation problem minimizes the sum of a loss function and a penalty term. The penalty term is usually weighted by a tuning parameter and encourages certain constraints on the parameters to be estimated. Particular choices of constraints lead to the popular lasso, fused-lasso, and other generalized $l_1$ penalized regression methods. Although there has been a lot of research in this area, developing efficient optimization methods for many nonseparable penalties remains a challenge. In this article we propose an exact path solver based on ordinary differential equations (EPSODE) that works for any convex loss function and can deal with generalized $l_1$ penalties as well as more complicated regularization such as inequality constraints encountered in shape-restricted regressions and nonparametric density estimation. In the path following process, the solution path hits, exits, and slides along the various constraints and vividly illustrates the tradeoffs between goodness of fit and model parsimony. In practice, the EPSODE can be coupled with AIC, BIC, $C_p$ or cross-validation to select an optimal tuning parameter. Our applications to generalized $l_1$ regularized generalized linear models, shape-restricted regressions, Gaussian graphical models, and nonparametric density estimation showcase the potential of the EPSODE algorithm.Comment: 28 pages, 5 figure

Prospects for Measuring Differential Rotation in White Dwarfs Through Asteroseismology

We examine the potential of asteroseismology for exploring the internal rotation of white dwarf stars. Data from global observing campaigns have revealed a wealth of frequencies, some of which show the signature of rotational splitting. Tools developed for helioseismology to use many solar p-mode frequencies for inversion of the rotation rate with depth are adapted to the case of more limited numbers of modes of low degree. We find that the small number of available modes in white dwarfs, coupled with the similarity between the rotational-splitting kernels of the modes, renders direct inversion unstable. Accordingly, we adopt what we consider to be plausible functional forms for the differential rotation profile; this is sufficiently restrictive to enable us to carry out a useful calibration. We show examples of this technique for PG 1159 stars and pulsating DB white dwarfs. Published frequency splittings for white dwarfs are currently not accurate enough for meaningful inversions; reanalysis of existing data can provide splittings of sufficient accuracy when the frequencies of individual peaks are extracted via least-squares fitting or multipeak decompositions. We find that when mode trapping is evident in the period spacing of g modes, the measured splittings can constrain dOmega/dr.Comment: 26 pages, 20 postscript figures. Accepted for publication in The Astrophysical Journa

Computational Methods for Sparse Solution of Linear Inverse Problems

The goal of the sparse approximation problem is to approximate a target signal using a linear combination of a few elementary signals drawn from a fixed collection. This paper surveys the major practical algorithms for sparse approximation. Specific attention is paid to computational issues, to the circumstances in which individual methods tend to perform well, and to the theoretical guarantees available. Many fundamental questions in electrical engineering, statistics, and applied mathematics can be posed as sparse approximation problems, making these algorithms versatile and relevant to a plethora of applications