2,951 research outputs found
Forward-backward truncated Newton methods for convex composite optimization
This paper proposes two proximal Newton-CG methods for convex nonsmooth
optimization problems in composite form. The algorithms are based on a a
reformulation of the original nonsmooth problem as the unconstrained
minimization of a continuously differentiable function, namely the
forward-backward envelope (FBE). The first algorithm is based on a standard
line search strategy, whereas the second one combines the global efficiency
estimates of the corresponding first-order methods, while achieving fast
asymptotic convergence rates. Furthermore, they are computationally attractive
since each Newton iteration requires the approximate solution of a linear
system of usually small dimension
A Scalable and Extensible Framework for Superposition-Structured Models
In many learning tasks, structural models usually lead to better
interpretability and higher generalization performance. In recent years,
however, the simple structural models such as lasso are frequently proved to be
insufficient. Accordingly, there has been a lot of work on
"superposition-structured" models where multiple structural constraints are
imposed. To efficiently solve these "superposition-structured" statistical
models, we develop a framework based on a proximal Newton-type method.
Employing the smoothed conic dual approach with the LBFGS updating formula, we
propose a scalable and extensible proximal quasi-Newton (SEP-QN) framework.
Empirical analysis on various datasets shows that our framework is potentially
powerful, and achieves super-linear convergence rate for optimizing some
popular "superposition-structured" statistical models such as the fused sparse
group lasso
- …