3,517 research outputs found
Wasserstein Introspective Neural Networks
We present Wasserstein introspective neural networks (WINN) that are both a
generator and a discriminator within a single model. WINN provides a
significant improvement over the recent introspective neural networks (INN)
method by enhancing INN's generative modeling capability. WINN has three
interesting properties: (1) A mathematical connection between the formulation
of the INN algorithm and that of Wasserstein generative adversarial networks
(WGAN) is made. (2) The explicit adoption of the Wasserstein distance into INN
results in a large enhancement to INN, achieving compelling results even with a
single classifier --- e.g., providing nearly a 20 times reduction in model size
over INN for unsupervised generative modeling. (3) When applied to supervised
classification, WINN also gives rise to improved robustness against adversarial
examples in terms of the error reduction. In the experiments, we report
encouraging results on unsupervised learning problems including texture, face,
and object modeling, as well as a supervised classification task against
adversarial attacks.Comment: Accepted to CVPR 2018 (Oral
Outlier detection using distributionally robust optimization under the Wasserstein metric
We present a Distributionally Robust Optimization (DRO) approach to outlier detection in a linear regression setting, where the closeness of probability distributions is measured using the Wasserstein metric. Training samples contaminated with outliers skew the regression plane computed by least squares and thus impede outlier detection. Classical approaches, such as robust regression, remedy this problem by downweighting the contribution of atypical data points. In contrast, our Wasserstein DRO approach hedges against a family of distributions that are close to the empirical distribution. We show that the resulting formulation encompasses a class of models, which include the regularized Least Absolute Deviation (LAD) as a special case. We provide new insights into the regularization term and give guidance on the selection of the regularization coefficient from the standpoint of a confidence region. We establish two types of performance guarantees for the solution to our formulation under mild conditions. One is related to its out-of-sample behavior, and the other concerns the discrepancy between the estimated and true regression planes. Extensive numerical results demonstrate the superiority of our approach to both robust regression and the regularized LAD in terms of estimation accuracy and outlier detection rates
Fair and Optimal Classification via Transports to Wasserstein-Barycenter
Fairness in automated decision-making systems has gained increasing attention
as their applications expand to real-world high-stakes domains. To facilitate
the design of fair ML systems, it is essential to understand the potential
trade-offs between fairness and predictive power, and the construction of the
optimal predictor under a given fairness constraint. In this paper, for general
classification problems under the group fairness criterion of demographic
parity (DP), we precisely characterize the trade-off between DP and
classification accuracy, referred to as the minimum cost of fairness. Our
insight comes from the key observation that finding the optimal fair classifier
is equivalent to solving a Wasserstein-barycenter problem under -norm
restricted to the vertices of the probability simplex. Inspired by our
characterization, we provide a construction of an optimal fair classifier
achieving this minimum cost via the composition of the Bayes regressor and
optimal transports from its output distributions to the barycenter. Our
construction naturally leads to an algorithm for post-processing any
pre-trained predictor to satisfy DP fairness, complemented with finite sample
guarantees. Experiments on real-world datasets verify and demonstrate the
effectiveness of our approaches.Comment: Code is at https://github.com/rxian/fair-classificatio
Matching Code and Law: Achieving Algorithmic Fairness with Optimal Transport
Increasingly, discrimination by algorithms is perceived as a societal and
legal problem. As a response, a number of criteria for implementing algorithmic
fairness in machine learning have been developed in the literature. This paper
proposes the Continuous Fairness Algorithm (CFA) which enables a
continuous interpolation between different fairness definitions. More
specifically, we make three main contributions to the existing literature.
First, our approach allows the decision maker to continuously vary between
specific concepts of individual and group fairness. As a consequence, the
algorithm enables the decision maker to adopt intermediate ``worldviews'' on
the degree of discrimination encoded in algorithmic processes, adding nuance to
the extreme cases of ``we're all equal'' (WAE) and ``what you see is what you
get'' (WYSIWYG) proposed so far in the literature. Second, we use optimal
transport theory, and specifically the concept of the barycenter, to maximize
decision maker utility under the chosen fairness constraints. Third, the
algorithm is able to handle cases of intersectionality, i.e., of
multi-dimensional discrimination of certain groups on grounds of several
criteria. We discuss three main examples (credit applications; college
admissions; insurance contracts) and map out the legal and policy implications
of our approach. The explicit formalization of the trade-off between individual
and group fairness allows this post-processing approach to be tailored to
different situational contexts in which one or the other fairness criterion may
take precedence. Finally, we evaluate our model experimentally.Comment: Vastly extended new version, now including computational experiment
- …