Twice epi-differentiability of a class of non-amenable composite functions

This paper focuses on the twice epi-differentiability of a class of non-amenable functions, which are the composition of a piecewise twice differentiable (PWTD) function and a parabolically semidifferentiable mapping. Such composite functions frequently appear those optimization problems covering major classes of constrained and composite optimization problems, as well as encompassing disjunctive programs and low-rank or/and sparsity optimization problems. To achieve the goal, we first justify the proper twice epi-differentiability, parabolic epi-differentiability and regularity of PWTD functions, and derive an upper and lower estimate for the second subderivatives of this class of composite functions in terms of a chain rule of their parabolic subderivatives. Then, we employ the obtained upper and lower estimates to characterize the parabolic regularity and second subderivatives and then achieve the proper twice epi-differentiability for several classes of popular functions, which include the compositions of PWTD outer functions and twice differentiable inner mappings, the regularized functions inducing group sparsity, and the indicator functions of the $q(q>1)$-order cone.Comment: 39page

On Choosing Initial Values of Iteratively Reweighted ℓ1\ell_1 Algorithms for the Piece-wise Exponential Penalty

Computing the proximal operator of the sparsity-promoting piece-wise exponential (PiE) penalty $1-e^{-|x|/\sigma}$ with a given shape parameter $\sigma>0$, which is treated as a popular nonconvex surrogate of $\ell_0$-norm, is fundamental in feature selection via support vector machines, image reconstruction, zero-one programming problems, compressed sensing, etc. Due to the nonconvexity of PiE, for a long time, its proximal operator is frequently evaluated via an iteratively reweighted $\ell_1$ algorithm, which substitutes PiE with its first-order approximation, however, the obtained solutions only are the critical point. Based on the exact characterization of the proximal operator of PiE, we explore how the iteratively reweighted $\ell_1$ solution deviates from the true proximal operator in certain regions, which can be explicitly identified in terms of $\sigma$, the initial value and the regularization parameter in the definition of the proximal operator. Moreover, the initial value can be adaptively and simply chosen to ensure that the iteratively reweighted $\ell_1$ solution belongs to the proximal operator of PiE