1,859 research outputs found
Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications
We propose a simple yet powerful test statistic to quantify the discrepancy
between two conditional distributions. The new statistic avoids the explicit
estimation of the underlying distributions in highdimensional space and it
operates on the cone of symmetric positive semidefinite (SPS) matrix using the
Bregman matrix divergence. Moreover, it inherits the merits of the correntropy
function to explicitly incorporate high-order statistics in the data. We
present the properties of our new statistic and illustrate its connections to
prior art. We finally show the applications of our new statistic on three
different machine learning problems, namely the multi-task learning over
graphs, the concept drift detection, and the information-theoretic feature
selection, to demonstrate its utility and advantage. Code of our statistic is
available at https://bit.ly/BregmanCorrentropy.Comment: manuscript accepted at IJCAI 20; added additional notes on
computational complexity and auto-differentiable property; code is available
at https://github.com/SJYuCNEL/Bregman-Correntropy-Conditional-Divergenc
Convex Optimization for Binary Classifier Aggregation in Multiclass Problems
Multiclass problems are often decomposed into multiple binary problems that
are solved by individual binary classifiers whose results are integrated into a
final answer. Various methods, including all-pairs (APs), one-versus-all (OVA),
and error correcting output code (ECOC), have been studied, to decompose
multiclass problems into binary problems. However, little study has been made
to optimally aggregate binary problems to determine a final answer to the
multiclass problem. In this paper we present a convex optimization method for
an optimal aggregation of binary classifiers to estimate class membership
probabilities in multiclass problems. We model the class membership probability
as a softmax function which takes a conic combination of discrepancies induced
by individual binary classifiers, as an input. With this model, we formulate
the regularized maximum likelihood estimation as a convex optimization problem,
which is solved by the primal-dual interior point method. Connections of our
method to large margin classifiers are presented, showing that the large margin
formulation can be considered as a limiting case of our convex formulation.
Numerical experiments on synthetic and real-world data sets demonstrate that
our method outperforms existing aggregation methods as well as direct methods,
in terms of the classification accuracy and the quality of class membership
probability estimates.Comment: Appeared in Proceedings of the 2014 SIAM International Conference on
Data Mining (SDM 2014
- …