Inertial Stochastic PALM (iSPALM) and Applications in Machine Learning

Inertial algorithms for minimizing nonsmooth and nonconvex functions as the inertial proximal alternating linearized minimization algorithm (iPALM) have demonstrated their superiority with respect to computation time over their non inertial variants. In many problems in imaging and machine learning, the objective functions have a special form involving huge data which encourage the application of stochastic algorithms. While algorithms based on stochastic gradient descent are still used in the majority of applications, recently also stochastic algorithms for minimizing nonsmooth and nonconvex functions were proposed. In this paper, we derive an inertial variant of a stochastic PALM algorithm with variance-reduced gradient estimator, called iSPALM, and prove linear convergence of the algorithm under certain assumptions. Our inertial approach can be seen as generalization of momentum methods widely used to speed up and stabilize optimization algorithms, in particular in machine learning, to nonsmooth problems. Numerical experiments for learning the weights of a so-called proximal neural network and the parameters of Student-t mixture models show that our new algorithm outperforms both stochastic PALM and its deterministic counterparts

A note on the dual treatment of higher order regularization functionals

In this paper, we apply the dual approach developed by A. Chambolle for the Rudin-Osher-Fatemi model to regularization functionals with higher order derivatives. We emphasize the linear algebra point of view by consequently using matrix-vector notation. Numerical examples demonstrate the differences between various second order regularization approaches

Nonlocal Myriad Filters for Cauchy Noise Removal

The contribution of this paper is two-fold. First, we introduce a generalized myriad filter, which is a method to compute the joint maximum likelihood estimator of the location and the scale parameter of the Cauchy distribution. Estimating only the location parameter is known as myriad filter. We propose an efficient algorithm to compute the generalized myriad filter and prove its convergence. Special cases of this algorithm result in the classical myriad filtering, respective an algorithm for estimating only the scale parameter. Based on an asymptotic analysis, we develop a second, even faster generalized myriad filtering technique. Second, we use our new approaches within a nonlocal, fully unsupervised method to denoise images corrupted by Cauchy noise. Special attention is paid to the determination of similar patches in noisy images. Numerical examples demonstrate the excellent performance of our algorithms which have moreover the advantage to be robust with respect to the parameter choice

Unsupervised Multi Class Segmentation of 3D Images with Intensity Inhomogeneities

Intensity inhomogeneities in images constitute a considerable challenge in image segmentation. In this paper we propose a novel biconvex variational model to tackle this task. We combine a total variation approach for multi class segmentation with a multiplicative model to handle the inhomogeneities. Our method assumes that the image intensity is the product of a smoothly varying part and a component which resembles important image structures such as edges. Therefore, we penalize in addition to the total variation of the label assignment matrix a quadratic difference term to cope with the smoothly varying factor. A critical point of our biconvex functional is computed by a modified proximal alternating linearized minimization method (PALM). We show that the assumptions for the convergence of the algorithm are fulfilled by our model. Various numerical examples demonstrate the very good performance of our method. Particular attention is paid to the segmentation of 3D FIB tomographical images which was indeed the motivation of our work