Super-Linear Convergence of Dual Augmented-Lagrangian Algorithm for Sparsity Regularized Estimation

We analyze the convergence behaviour of a recently proposed algorithm for regularized estimation called Dual Augmented Lagrangian (DAL). Our analysis is based on a new interpretation of DAL as a proximal minimization algorithm. We theoretically show under some conditions that DAL converges super-linearly in a non-asymptotic and global sense. Due to a special modelling of sparse estimation problems in the context of machine learning, the assumptions we make are milder and more natural than those made in conventional analysis of augmented Lagrangian algorithms. In addition, the new interpretation enables us to generalize DAL to wide varieties of sparse estimation problems. We experimentally confirm our analysis in a large scale $\ell_1$-regularized logistic regression problem and extensively compare the efficiency of DAL algorithm to previously proposed algorithms on both synthetic and benchmark datasets.Comment: 51 pages, 9 figure

Semi-Supervised Kernel PCA

We present three generalisations of Kernel Principal Components Analysis (KPCA) which incorporate knowledge of the class labels of a subset of the data points. The first, MV-KPCA, penalises within class variances similar to Fisher discriminant analysis. The second, LSKPCA is a hybrid of least squares regression and kernel PCA. The final LR-KPCA is an iteratively reweighted version of the previous which achieves a sigmoid loss function on the labeled points. We provide a theoretical risk bound as well as illustrative experiments on real and toy data sets

Peridynamic Galerkin methods for nonlinear solid mechanics

Simulation-driven product development is nowadays an essential part in the industrial digitalization. Notably, there is an increasing interest in realistic high-fidelity simulation methods in the fast-growing field of additive and ablative manufacturing processes. Thanks to their flexibility, meshfree solution methods are particularly suitable for simulating the stated processes, often accompanied by large deformations, variable discontinuities, or phase changes. Furthermore, in the industrial domain, the meshing of complex geometries represents a significant workload, which is usually minor for meshfree methods. Over the years, several meshfree schemes have been developed. Nevertheless, along with their flexibility in discretization, meshfree methods often endure a decrease in accuracy, efficiency and stability or suffer from a significantly increased computation time. Peridynamics is an alternative theory to local continuum mechanics for describing partial differential equations in a non-local integro-differential form. The combination of the so-called peridynamic correspondence formulation with a particle discretization yields a flexible meshfree simulation method, though does not lead to reliable results without further treatment.\newline In order to develop a reliable, robust and still flexible meshfree simulation method, the classical correspondence formulation is generalized into the Peridynamic Galerkin (PG) methods in this work. On this basis, conditions on the meshfree shape functions of virtual and actual displacement are presented, which allow an accurate imposition of force and displacement boundary conditions and lead to stability and optimal convergence rates. Based on Taylor expansions moving with the evaluation point, special shape functions are introduced that satisfy all the previously mentioned requirements employing correction schemes. In addition to displacement-based formulations, a variety of stabilized, mixed and enriched variants are developed, which are tailored in their application to the nearly incompressible and elasto-plastic finite deformation of solids, highlighting the broad design scope within the PG methods. Extensive numerical validations and benchmark simulations are performed to show the impact of violating different shape function requirements as well as demonstrating the properties of the different PG formulations. Compared to related Finite Element formulations, the PG methods exhibit similar convergence properties. Furthermore, an increased computation time due to non-locality is counterbalanced by a considerably improved robustness against poorly meshed discretizations

A finite membrane element formulation for surfactants

Surfactants play an important role in various physiological and biomechanical applications. An example is the respiratory system, where pulmonary surfactants facilitate the breathing and reduce the possibility of airway blocking by lowering the surface tension when the lung volume decreases during exhalation. This function is due to the dynamic surface tension of pulmonary surfactants, which depends on the concentration of surfactants spread on the liquid layer lining the interior surface of the airways and alveoli. Here, a finite membrane element formulation for liquids is introduced that allows for the dynamics of concentration-dependent surface tension, as is the particular case for pulmonary surfactants. A straightforward approach is suggested to model the contact line between liquid drops/menisci and planar solid substrates, which allows the presented framework to be easily used for drop shape analysis. It is further shown how line tension can be taken into account. Following an isogeometric approach, NURBS-based finite elements are used for the discretization of the membrane surface. The capabilities of the presented computational model is demonstrated by different numerical examples - such as the simulation of liquid films, constrained and unconstrained sessile drops, pendant drops and liquid bridges - and the results are compared with experimental data.Comment: Some typos are removed. Eqs. 13 and 105 are modified. Eqs. 64 and 73 are added; thus, the rest of equations is renumbered. All the numerical experiments are repeated. The example of Sec. 6.3 is slightly modifie

Forward-backward truncated Newton methods for convex composite optimization

This paper proposes two proximal Newton-CG methods for convex nonsmooth optimization problems in composite form. The algorithms are based on a a reformulation of the original nonsmooth problem as the unconstrained minimization of a continuously differentiable function, namely the forward-backward envelope (FBE). The first algorithm is based on a standard line search strategy, whereas the second one combines the global efficiency estimates of the corresponding first-order methods, while achieving fast asymptotic convergence rates. Furthermore, they are computationally attractive since each Newton iteration requires the approximate solution of a linear system of usually small dimension