On Newton Screening

Screening and working set techniques are important approaches to reducing the size of an optimization problem. They have been widely used in accelerating first-order methods for solving large-scale sparse learning problems. In this paper, we develop a new screening method called Newton screening (NS) which is a generalized Newton method with a built-in screening mechanism. We derive an equivalent KKT system for the Lasso and utilize a generalized Newton method to solve the KKT equations. Based on this KKT system, a built-in working set with a relatively small size is first determined using the sum of primal and dual variables generated from the previous iteration, then the primal variable is updated by solving a least-squares problem on the working set and the dual variable updated based on a closed-form expression. Moreover, we consider a sequential version of Newton screening (SNS) with a warm-start strategy. We show that NS possesses an optimal convergence property in the sense that it achieves one-step local convergence. Under certain regularity conditions on the feature matrix, we show that SNS hits a solution with the same signs as the underlying true target and achieves a sharp estimation error bound with high probability. Simulation studies and real data analysis support our theoretical results and demonstrate that SNS is faster and more accurate than several state-of-the-art methods in our comparative studies

Spectral operators of matrices: Semismoothness and characterizations of the generalized jacobian

Spectral operators of matrices proposed recently in [C. Ding, D. F. Sun, J. Sun, and K. C. Toh, Math. Program., 168 (2018), pp. 509{531] are a class of matrix-valued functions, which map matrices to matrices by applying a vector-to-vector function to all eigenvalues/singular values of the underlying matrices. Spectral operators play a crucial role in the study of various applications involving matrices such as matrix optimization problems that include semidefinite programming as one of most important example classes. In this paper, we will study more fundamental first- and second-order properties of spectral operators, including the Lipschitz continuity, ρ-order B(ouligand)-differentiability (0 < ρ≤ 1), ρ-order G-semismoothness (0 < ρ≤ 1), and characteriza- tion of generalized Jacobians

An efficient semismooth Newton based algorithm for convex clustering

35th International Conference on Machine Learning, ICML 2018, Stockholm, Sweden, 10-15 July 2018202305 bcchVersion of RecordSelf-fundedPublishe