5,196 research outputs found
Curating Naturally Adversarial Datasets for Trustworthy AI in Healthcare
Deep learning models have shown promising predictive accuracy for time-series
healthcare applications. However, ensuring the robustness of these models is
vital for building trustworthy AI systems. Existing research predominantly
focuses on robustness to synthetic adversarial examples, crafted by adding
imperceptible perturbations to clean input data. However, these synthetic
adversarial examples do not accurately reflect the most challenging real-world
scenarios, especially in the context of healthcare data. Consequently,
robustness to synthetic adversarial examples may not necessarily translate to
robustness against naturally occurring adversarial examples, which is highly
desirable for trustworthy AI. We propose a method to curate datasets comprised
of natural adversarial examples to evaluate model robustness. The method relies
on probabilistic labels obtained from automated weakly-supervised labeling that
combines noisy and cheap-to-obtain labeling heuristics. Based on these labels,
our method adversarially orders the input data and uses this ordering to
construct a sequence of increasingly adversarial datasets. Our evaluation on
six medical case studies and three non-medical case studies demonstrates the
efficacy and statistical validity of our approach to generating naturally
adversarial dataset
Robust Multilingual Part-of-Speech Tagging via Adversarial Training
Adversarial training (AT) is a powerful regularization method for neural
networks, aiming to achieve robustness to input perturbations. Yet, the
specific effects of the robustness obtained from AT are still unclear in the
context of natural language processing. In this paper, we propose and analyze a
neural POS tagging model that exploits AT. In our experiments on the Penn
Treebank WSJ corpus and the Universal Dependencies (UD) dataset (27 languages),
we find that AT not only improves the overall tagging accuracy, but also 1)
prevents over-fitting well in low resource languages and 2) boosts tagging
accuracy for rare / unseen words. We also demonstrate that 3) the improved
tagging performance by AT contributes to the downstream task of dependency
parsing, and that 4) AT helps the model to learn cleaner word representations.
5) The proposed AT model is generally effective in different sequence labeling
tasks. These positive results motivate further use of AT for natural language
tasks.Comment: NAACL 201
- …