98 research outputs found

    Generalization and Equilibrium in Generative Adversarial Nets (GANs)

    Full text link
    We show that training of generative adversarial network (GAN) may not have good generalization properties; e.g., training may appear successful but the trained distribution may be far from target distribution in standard metrics. However, generalization does occur for a weaker metric called neural net distance. It is also shown that an approximate pure equilibrium exists in the discriminator/generator game for a special class of generators with natural training objectives when generator capacity and training set sizes are moderate. This existence of equilibrium inspires MIX+GAN protocol, which can be combined with any existing GAN training, and empirically shown to improve some of them.Comment: This is an updated version of an ICML'17 paper with the same title. The main difference is that in the ICML'17 version the pure equilibrium result was only proved for Wasserstein GAN. In the current version the result applies to most reasonable training objectives. In particular, Theorem 4.3 now applies to both original GAN and Wasserstein GA

    Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy

    Full text link
    Posterior sampling, i.e., exponential mechanism to sample from the posterior distribution, provides ε\varepsilon-pure differential privacy (DP) guarantees and does not suffer from potentially unbounded privacy breach introduced by (ε,δ)(\varepsilon,\delta)-approximate DP. In practice, however, one needs to apply approximate sampling methods such as Markov chain Monte Carlo (MCMC), thus re-introducing the unappealing δ\delta-approximation error into the privacy guarantees. To bridge this gap, we propose the Approximate SAample Perturbation (abbr. ASAP) algorithm which perturbs an MCMC sample with noise proportional to its Wasserstein-infinity (W∞W_\infty) distance from a reference distribution that satisfies pure DP or pure Gaussian DP (i.e., δ=0\delta=0). We then leverage a Metropolis-Hastings algorithm to generate the sample and prove that the algorithm converges in W∞_\infty distance. We show that by combining our new techniques with a careful localization step, we obtain the first nearly linear-time algorithm that achieves the optimal rates in the DP-ERM problem with strongly convex and smooth losses

    A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

    Full text link
    Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features. This paper introduces a la carte embedding, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings. Our method relies mainly on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. This transform is applicable on the fly in the future when a new text feature or rare word is encountered, even if only a single usage example is available. We introduce a new dataset showing how the a la carte method requires fewer examples of words in context to learn high-quality embeddings and we obtain state-of-the-art results on a nonce task and some unsupervised document classification tasks.Comment: 11 pages, 2 figures, To appear in ACL 201

    A La Carte Embedding: Cheap but Effective Induction of Semantic Feature Vectors

    Full text link
    Motivations like domain adaptation, transfer learning, and feature learning have fueled interest in inducing embeddings for rare or unseen words, n-grams, synsets, and other textual features. This paper introduces a la carte embedding, a simple and general alternative to the usual word2vec-based approaches for building such representations that is based upon recent theoretical results for GloVe-like embeddings. Our method relies mainly on a linear transformation that is efficiently learnable using pretrained word vectors and linear regression. This transform is applicable on the fly in the future when a new text feature or rare word is encountered, even if only a single usage example is available. We introduce a new dataset showing how the a la carte method requires fewer examples of words in context to learn high-quality embeddings and we obtain state-of-the-art results on a nonce task and some unsupervised document classification tasks.Comment: 11 pages, 2 figures, To appear in ACL 201

    Analysis of Prognostic Risk Factors Determining Poor Functional Recovery After Comprehensive Rehabilitation Including Motor-Imagery Brain-Computer Interface Training in Stroke Patients: A Prospective Study

    Get PDF
    Objective: Upper limb (UL) motor function recovery, especially distal function, is one of the main goals of stroke rehabilitation as this function is important to perform activities of daily living (ADL). The efficacy of the motor-imagery brain-computer interface (MI-BCI) has been demonstrated in patients with stroke. Most patients with stroke receive comprehensive rehabilitation, including MI-BCI and routine training. However, most aspects of MI-BCI training for patients with subacute stroke are based on routine training. Risk factors for inadequate distal UL functional recovery in these patients remain unclear; therefore, it is more realistic to explore the prognostic factors of this comprehensive treatment based on clinical practice. The present study aims to investigate the independent risk factors that might lead to inadequate distal UL functional recovery in patients with stroke after comprehensive rehabilitation including MI-BCI (CRIMI-BCI).Methods: This prospective study recruited 82 patients with stroke who underwent CRIMI-BCI. Motor-imagery brain-computer interface training was performed for 60 min per day, 5 days per week for 4 weeks. The primary outcome was improvement of the wrist and hand dimensionality of Fugl-Meyer Assessment (δFMA-WH). According to the improvement score, the patients were classified into the efficient group (EG, δFMA-WH > 2) and the inefficient group (IG, δFMA-WH ≤ 2). Binary logistic regression was used to analyze clinical and demographic data, including aphasia, spasticity of the affected hand [assessed by Modified Ashworth Scale (MAS-H)], initial UL function, age, gender, time since stroke (TSS), lesion hemisphere, and lesion location.Results: Seventy-three patients completed the study. After training, all patients showed significant improvement in FMA-UL (Z = 7.381, p = 0.000**), FMA-SE (Z = 7.336, p = 0.000**), and FMA-WH (Z = 6.568, p = 0.000**). There were 35 patients (47.9%) in the IG group and 38 patients (52.1%) in the EG group. Multivariate analysis revealed that presence of aphasia [odds ratio (OR) 4.617, 95% confidence interval (CI) 1.435–14.860; p < 0.05], initial FMA-UL score ≤ 30 (OR 5.158, 95% CI 1.150–23.132; p < 0.05), and MAS-H ≥ level I+ (OR 3.810, 95% CI 1.231–11.790; p < 0.05) were the risk factors for inadequate distal UL functional recovery in patients with stroke after CRIMI-BCI.Conclusion: We concluded that CRIMI-BCI improved UL function in stroke patients with varying effectiveness. Inferior initial UL function, significant hand spasticity, and presence of aphasia were identified as independent risk factors for inadequate distal UL functional recovery in stroke patients after CRIMI-BCI
    • …
    corecore