136 research outputs found
Revisiting classifier two-sample tests
International audienceThe goal of two-sample tests is to assess whether two samples, and , are drawn from the same distribution. Perhaps intriguingly, one relatively unexplored method to build two-sample tests is the use of binary classifiers. In particular, construct a dataset by pairing the n examples in S P with a positive label, and by pairing the m examples in with a negative label. If the null hypothesis " " is true, then the classification accuracy of a binary classifier on a held-out subset of this dataset should remain near chance-level. As we will show, such Classifier Two-Sample Tests (C2ST) learn a suitable representation of the data on the fly, return test statistics in interpretable units, have a simple null distribution, and their predictive uncertainty allow to interpret where P and Q differ. The goal of this paper is to establish the properties, performance, and uses of C2ST. First, we analyze their main theoretical properties. Second, we compare their performance against a variety of state-of-the-art alternatives. Third, we propose their use to evaluate the sample quality of generative models with intractable likelihoods, such as Generative Adversarial Networks (GANs). Fourth, we showcase the novel application of GANs together with C2ST for causal discovery
Revisiting Precision and Recall Definition for Generative Model Evaluation
In this article we revisit the definition of Precision-Recall (PR) curves for
generative models proposed by Sajjadi et al. (arXiv:1806.00035). Rather than
providing a scalar for generative quality, PR curves distinguish mode-collapse
(poor recall) and bad quality (poor precision). We first generalize their
formulation to arbitrary measures, hence removing any restriction to finite
support. We also expose a bridge between PR curves and type I and type II error
rates of likelihood ratio classifiers on the task of discriminating between
samples of the two distributions. Building upon this new perspective, we
propose a novel algorithm to approximate precision-recall curves, that shares
some interesting methodological properties with the hypothesis testing
technique from Lopez-Paz et al (arXiv:1610.06545). We demonstrate the interest
of the proposed formulation over the original approach on controlled
multi-modal datasets.Comment: ICML 201
PathologyGAN: Learning deep representations of cancer tissue
We apply Generative Adversarial Networks (GANs) to the domain of digital
pathology. Current machine learning research for digital pathology focuses on
diagnosis, but we suggest a different approach and advocate that generative
models could drive forward the understanding of morphological characteristics
of cancer tissue. In this paper, we develop a framework which allows GANs to
capture key tissue features and uses these characteristics to give structure to
its latent space. To this end, we trained our model on 249K H&E breast cancer
tissue images, extracted from 576 TMA images of patients from the Netherlands
Cancer Institute (NKI) and Vancouver General Hospital (VGH) cohorts. We show
that our model generates high quality images, with a Frechet Inception Distance
(FID) of 16.65. We further assess the quality of the images with cancer tissue
characteristics (e.g. count of cancer, lymphocytes, or stromal cells), using
quantitative information to calculate the FID and showing consistent
performance of 9.86. Additionally, the latent space of our model shows an
interpretable structure and allows semantic vector operations that translate
into tissue feature transformations. Furthermore, ratings from two expert
pathologists found no significant difference between our generated tissue
images from real ones. The code, generated images, and pretrained model are
available at https://github.com/AdalbertoCq/Pathology-GANComment: MIDL 2020 final versio
- …