Robust and Sparse Regression via γ\gamma-divergence

In high-dimensional data, many sparse regression methods have been proposed. However, they may not be robust against outliers. Recently, the use of density power weight has been studied for robust parameter estimation and the corresponding divergences have been discussed. One of such divergences is the $\gamma$-divergence and the robust estimator using the $\gamma$-divergence is known for having a strong robustness. In this paper, we consider the robust and sparse regression based on $\gamma$-divergence. We extend the $\gamma$-divergence to the regression problem and show that it has a strong robustness under heavy contamination even when outliers are heterogeneous. The loss function is constructed by an empirical estimate of the $\gamma$-divergence with sparse regularization and the parameter estimate is defined as the minimizer of the loss function. To obtain the robust and sparse estimate, we propose an efficient update algorithm which has a monotone decreasing property of the loss function. Particularly, we discuss a linear regression problem with $L_1$ regularization in detail. In numerical experiments and real data analyses, we see that the proposed method outperforms past robust and sparse methods.Comment: 25 page

Postquantum Br\`{e}gman relative entropies and nonlinear resource theories

We introduce the family of postquantum Br\`{e}gman relative entropies, based on nonlinear embeddings into reflexive Banach spaces (with examples given by reflexive noncommutative Orlicz spaces over semi-finite W*-algebras, nonassociative L$_p$ spaces over semi-finite JBW-algebras, and noncommutative L$_p$ spaces over arbitrary W*-algebras). This allows us to define a class of geometric categories for nonlinear postquantum inference theory (providing an extension of Chencov's approach to foundations of statistical inference), with constrained maximisations of Br\`{e}gman relative entropies as morphisms and nonlinear images of closed convex sets as objects. Further generalisation to a framework for nonlinear convex operational theories is developed using a larger class of morphisms, determined by Br\`{e}gman nonexpansive operations (which provide a well-behaved family of Mielnik's nonlinear transmitters). As an application, we derive a range of nonlinear postquantum resource theories determined in terms of this class of operations.Comment: v2: several corrections and improvements, including an extension to the postquantum (generally) and JBW-algebraic (specifically) cases, a section on nonlinear resource theories, and more informative paper's titl

Extension of information geometry for modelling non-statistical systems

In this dissertation, an abstract formalism extending information geometry is introduced. This framework encompasses a broad range of modelling problems, including possible applications in machine learning and in the information theoretical foundations of quantum theory. Its purely geometrical foundations make no use of probability theory and very little assumptions about the data or the models are made. Starting only from a divergence function, a Riemannian geometrical structure consisting of a metric tensor and an affine connection is constructed and its properties are investigated. Also the relation to information geometry and in particular the geometry of exponential families of probability distributions is elucidated. It turns out this geometrical framework offers a straightforward way to determine whether or not a parametrised family of distributions can be written in exponential form. Apart from the main theoretical chapter, the dissertation also contains a chapter of examples illustrating the application of the formalism and its geometric properties, a brief introduction to differential geometry and a historical overview of the development of information geometry.Comment: PhD thesis, University of Antwerp, Advisors: Prof. dr. Jan Naudts and Prof. dr. Jacques Tempere, December 2014, 108 page