27,428 research outputs found
Generative and Discriminative Text Classification with Recurrent Neural Networks
We empirically characterize the performance of discriminative and generative
LSTM models for text classification. We find that although RNN-based generative
models are more powerful than their bag-of-words ancestors (e.g., they account
for conditional dependencies across words in a document), they have higher
asymptotic error rates than discriminatively trained RNN models. However we
also find that generative models approach their asymptotic error rate more
rapidly than their discriminative counterparts---the same pattern that Ng &
Jordan (2001) proved holds for linear classification models that make more
naive conditional independence assumptions. Building on this finding, we
hypothesize that RNN-based generative classification models will be more robust
to shifts in the data distribution. This hypothesis is confirmed in a series of
experiments in zero-shot and continual learning settings that show that
generative models substantially outperform discriminative models
Generative-Discriminative Complementary Learning
Majority of state-of-the-art deep learning methods are discriminative
approaches, which model the conditional distribution of labels given inputs
features. The success of such approaches heavily depends on high-quality
labeled instances, which are not easy to obtain, especially as the number of
candidate classes increases. In this paper, we study the complementary learning
problem. Unlike ordinary labels, complementary labels are easy to obtain
because an annotator only needs to provide a yes/no answer to a randomly chosen
candidate class for each instance. We propose a generative-discriminative
complementary learning method that estimates the ordinary labels by modeling
both the conditional (discriminative) and instance (generative) distributions.
Our method, we call Complementary Conditional GAN (CCGAN), improves the
accuracy of predicting ordinary labels and can generate high-quality instances
in spite of weak supervision. In addition to the extensive empirical studies,
we also theoretically show that our model can retrieve the true conditional
distribution from the complementarily-labeled data
Generative discriminative models for multivariate inference and statistical mapping in medical imaging
This paper presents a general framework for obtaining interpretable
multivariate discriminative models that allow efficient statistical inference
for neuroimage analysis. The framework, termed generative discriminative
machine (GDM), augments discriminative models with a generative regularization
term. We demonstrate that the proposed formulation can be optimized in closed
form and in dual space, allowing efficient computation for high dimensional
neuroimaging datasets. Furthermore, we provide an analytic estimation of the
null distribution of the model parameters, which enables efficient statistical
inference and p-value computation without the need for permutation testing. We
compared the proposed method with both purely generative and discriminative
learning methods in two large structural magnetic resonance imaging (sMRI)
datasets of Alzheimer's disease (AD) (n=415) and Schizophrenia (n=853). Using
the AD dataset, we demonstrated the ability of GDM to robustly handle
confounding variations. Using Schizophrenia dataset, we demonstrated the
ability of GDM to handle multi-site studies. Taken together, the results
underline the potential of the proposed approach for neuroimaging analyses.Comment: To appear in MICCAI 2018 proceeding
Generative Noisy-Label Learning by Implicit Dicriminative Approximation with Partial Label Prior
The learning with noisy labels has been addressed with both discriminative
and generative models. Although discriminative models have dominated the field
due to their simpler modeling and more efficient computational training
processes, generative models offer a more effective means of disentangling
clean and noisy labels and improving the estimation of the label transition
matrix. However, generative approaches maximize the joint likelihood of noisy
labels and data using a complex formulation that only indirectly optimizes the
model of interest associating data and clean labels. Additionally, these
approaches rely on generative models that are challenging to train and tend to
use uninformative clean label priors. In this paper, we propose a new
generative noisy-label learning approach that addresses these three issues.
First, we propose a new model optimisation that directly associates data and
clean labels. Second, the generative model is implicitly estimated using a
discriminative model, eliminating the inefficient training of a generative
model. Third, we propose a new informative label prior inspired by partial
label learning as supervision signal for noisy label learning. Extensive
experiments on several noisy-label benchmarks demonstrate that our generative
model provides state-of-the-art results while maintaining a similar
computational complexity as discriminative models
- …