The Bernstein Function: A Unifying Framework of Nonconvex Penalization in Sparse Estimation

In this paper we study nonconvex penalization using Bernstein functions. Since the Bernstein function is concave and nonsmooth at the origin, it can induce a class of nonconvex functions for high-dimensional sparse estimation problems. We derive a threshold function based on the Bernstein penalty and give its mathematical properties in sparsity modeling. We show that a coordinate descent algorithm is especially appropriate for penalized regression problems with the Bernstein penalty. Additionally, we prove that the Bernstein function can be defined as the concave conjugate of a $\varphi$-divergence and develop a conjugate maximization algorithm for finding the sparse solution. Finally, we particularly exemplify a family of Bernstein nonconvex penalties based on a generalized Gamma measure and conduct empirical analysis for this family

The Matrix Ridge Approximation: Algorithms and Applications

We are concerned with an approximation problem for a symmetric positive semidefinite matrix due to motivation from a class of nonlinear machine learning methods. We discuss an approximation approach that we call {matrix ridge approximation}. In particular, we define the matrix ridge approximation as an incomplete matrix factorization plus a ridge term. Moreover, we present probabilistic interpretations using a normal latent variable model and a Wishart model for this approximation approach. The idea behind the latent variable model in turn leads us to an efficient EM iterative method for handling the matrix ridge approximation problem. Finally, we illustrate the applications of the approximation approach in multivariate data analysis. Empirical studies in spectral clustering and Gaussian process regression show that the matrix ridge approximation with the EM iteration is potentially useful

Characterisation of matrix entropies

The notion of matrix entropy was introduced by Tropp and Chen with the aim of measuring the fluctuations of random matrices. It is a certain entropy functional constructed from a representing function with prescribed properties, and Tropp and Chen gave some examples. We give several abstract characterisations of matrix entropies together with a sufficient condition in terms of the second derivative of their representing function.Comment: Major revision. We found an error in the previous version that we cannot repair. It implies that we no longer can be certain that the sufficient condition of operator convexity of the second derivative of a matrix entropy is also necessary. We added more abstract characterisations of matrix entropies and improved the analysis of the concrete example

On the circumradius of a special class of n-simplices

An n-simplex is called circumscriptible (or edge-incentric) if there is a sphere tangent to all its n(n + 1)/2 edges. We obtain a closed formula for the radius of the circumscribed sphere of the circumscriptible n-simplex, and also prove a double inequality involving the circumradius and the edge-inradius of such simplices. Among this inequality settles affirmatively a part of a problem posed by the authors.Comment: 11 page

Efficient Algorithms and Error Analysis for the Modified Nystrom Method

Many kernel methods suffer from high time and space complexities and are thus prohibitive in big-data applications. To tackle the computational challenge, the Nystr\"om method has been extensively used to reduce time and space complexities by sacrificing some accuracy. The Nystr\"om method speedups computation by constructing an approximation of the kernel matrix using only a few columns of the matrix. Recently, a variant of the Nystr\"om method called the modified Nystr\"om method has demonstrated significant improvement over the standard Nystr\"om method in approximation accuracy, both theoretically and empirically. In this paper, we propose two algorithms that make the modified Nystr\"om method practical. First, we devise a simple column selection algorithm with a provable error bound. Our algorithm is more efficient and easier to implement than and nearly as accurate as the state-of-the-art algorithm. Second, with the selected columns at hand, we propose an algorithm that computes the approximation in lower time complexity than the approach in the previous work. Furthermore, we prove that the modified Nystr\"om method is exact under certain conditions, and we establish a lower error bound for the modified Nystr\"om method.Comment: 9-page paper plus appendix. In Proceedings of the 17th International Conference on Artificial Intelligence and Statistics (AISTATS) 2014, Reykjavik, Iceland. JMLR: W&CP volume 3