98 research outputs found
Generalization and Equilibrium in Generative Adversarial Nets (GANs)
We show that training of generative adversarial network (GAN) may not have
good generalization properties; e.g., training may appear successful but the
trained distribution may be far from target distribution in standard metrics.
However, generalization does occur for a weaker metric called neural net
distance. It is also shown that an approximate pure equilibrium exists in the
discriminator/generator game for a special class of generators with natural
training objectives when generator capacity and training set sizes are
moderate.
This existence of equilibrium inspires MIX+GAN protocol, which can be
combined with any existing GAN training, and empirically shown to improve some
of them.Comment: This is an updated version of an ICML'17 paper with the same title.
The main difference is that in the ICML'17 version the pure equilibrium
result was only proved for Wasserstein GAN. In the current version the result
applies to most reasonable training objectives. In particular, Theorem 4.3
now applies to both original GAN and Wasserstein GA
Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy
Posterior sampling, i.e., exponential mechanism to sample from the posterior
distribution, provides -pure differential privacy (DP) guarantees
and does not suffer from potentially unbounded privacy breach introduced by
-approximate DP. In practice, however, one needs to apply
approximate sampling methods such as Markov chain Monte Carlo (MCMC), thus
re-introducing the unappealing -approximation error into the privacy
guarantees. To bridge this gap, we propose the Approximate SAample Perturbation
(abbr. ASAP) algorithm which perturbs an MCMC sample with noise proportional to
its Wasserstein-infinity () distance from a reference distribution
that satisfies pure DP or pure Gaussian DP (i.e., ). We then leverage
a Metropolis-Hastings algorithm to generate the sample and prove that the
algorithm converges in W distance. We show that by combining our new
techniques with a careful localization step, we obtain the first nearly
linear-time algorithm that achieves the optimal rates in the DP-ERM problem
with strongly convex and smooth losses
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors
Motivations like domain adaptation, transfer learning, and feature learning
have fueled interest in inducing embeddings for rare or unseen words, n-grams,
synsets, and other textual features. This paper introduces a la carte
embedding, a simple and general alternative to the usual word2vec-based
approaches for building such representations that is based upon recent
theoretical results for GloVe-like embeddings. Our method relies mainly on a
linear transformation that is efficiently learnable using pretrained word
vectors and linear regression. This transform is applicable on the fly in the
future when a new text feature or rare word is encountered, even if only a
single usage example is available. We introduce a new dataset showing how the a
la carte method requires fewer examples of words in context to learn
high-quality embeddings and we obtain state-of-the-art results on a nonce task
and some unsupervised document classification tasks.Comment: 11 pages, 2 figures, To appear in ACL 201
A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors
Motivations like domain adaptation, transfer learning, and feature learning
have fueled interest in inducing embeddings for rare or unseen words, n-grams,
synsets, and other textual features. This paper introduces a la carte
embedding, a simple and general alternative to the usual word2vec-based
approaches for building such representations that is based upon recent
theoretical results for GloVe-like embeddings. Our method relies mainly on a
linear transformation that is efficiently learnable using pretrained word
vectors and linear regression. This transform is applicable on the fly in the
future when a new text feature or rare word is encountered, even if only a
single usage example is available. We introduce a new dataset showing how the a
la carte method requires fewer examples of words in context to learn
high-quality embeddings and we obtain state-of-the-art results on a nonce task
and some unsupervised document classification tasks.Comment: 11 pages, 2 figures, To appear in ACL 201
Analysis of Prognostic Risk Factors Determining Poor Functional Recovery After Comprehensive Rehabilitation Including Motor-Imagery Brain-Computer Interface Training in Stroke Patients: A Prospective Study
Objective: Upper limb (UL) motor function recovery, especially distal function, is one of the main goals of stroke rehabilitation as this function is important to perform activities of daily living (ADL). The efficacy of the motor-imagery brain-computer interface (MI-BCI) has been demonstrated in patients with stroke. Most patients with stroke receive comprehensive rehabilitation, including MI-BCI and routine training. However, most aspects of MI-BCI training for patients with subacute stroke are based on routine training. Risk factors for inadequate distal UL functional recovery in these patients remain unclear; therefore, it is more realistic to explore the prognostic factors of this comprehensive treatment based on clinical practice. The present study aims to investigate the independent risk factors that might lead to inadequate distal UL functional recovery in patients with stroke after comprehensive rehabilitation including MI-BCI (CRIMI-BCI).Methods: This prospective study recruited 82 patients with stroke who underwent CRIMI-BCI. Motor-imagery brain-computer interface training was performed for 60 min per day, 5 days per week for 4 weeks. The primary outcome was improvement of the wrist and hand dimensionality of Fugl-Meyer Assessment (δFMA-WH). According to the improvement score, the patients were classified into the efficient group (EG, δFMA-WH > 2) and the inefficient group (IG, δFMA-WH ≤ 2). Binary logistic regression was used to analyze clinical and demographic data, including aphasia, spasticity of the affected hand [assessed by Modified Ashworth Scale (MAS-H)], initial UL function, age, gender, time since stroke (TSS), lesion hemisphere, and lesion location.Results: Seventy-three patients completed the study. After training, all patients showed significant improvement in FMA-UL (Z = 7.381, p = 0.000**), FMA-SE (Z = 7.336, p = 0.000**), and FMA-WH (Z = 6.568, p = 0.000**). There were 35 patients (47.9%) in the IG group and 38 patients (52.1%) in the EG group. Multivariate analysis revealed that presence of aphasia [odds ratio (OR) 4.617, 95% confidence interval (CI) 1.435–14.860; p < 0.05], initial FMA-UL score ≤ 30 (OR 5.158, 95% CI 1.150–23.132; p < 0.05), and MAS-H ≥ level I+ (OR 3.810, 95% CI 1.231–11.790; p < 0.05) were the risk factors for inadequate distal UL functional recovery in patients with stroke after CRIMI-BCI.Conclusion: We concluded that CRIMI-BCI improved UL function in stroke patients with varying effectiveness. Inferior initial UL function, significant hand spasticity, and presence of aphasia were identified as independent risk factors for inadequate distal UL functional recovery in stroke patients after CRIMI-BCI
- …