89 research outputs found
Language Detoxification with Attribute-Discriminative Latent Space
Transformer-based Language Models (LMs) have achieved impressive results on
natural language understanding tasks, but they can also generate toxic text
such as insults, threats, and profanity, limiting their real-world
applications. To overcome this issue, a few text generation approaches aim to
detoxify toxic texts using additional LMs or perturbations. However, previous
methods require excessive memory, computations, and time which are serious
bottlenecks in their real-world application. To address such limitations, we
propose an effective yet efficient method for language detoxification using an
attribute-discriminative latent space. Specifically, we project the latent
space of an original Transformer LM onto a discriminative latent space that
well-separates texts by their attributes using a projection block and an
attribute discriminator. This allows the LM to control the text generation to
be non-toxic with minimal memory and computation overhead. We validate our
model, Attribute-Discriminative Language Model (ADLM) on detoxified language
and dialogue generation tasks, on which our method significantly outperforms
baselines both in performance and efficiency.Comment: ACL 2023; *Equal contribution. Author ordering determined by coin
fli
Re-Evaluating the Gender Gap: a Cross-Sectional analysis of accepted american academy of Neurology annual Meeting abstracts in 2020 and 2021
BACKGROUND AND OBJECTIVE: Prior studies reveal that invited speaker panels, editorial boards, authors of practice guidelines, and senior authors of published articles are disproportionately male in the neurology field. We aimed to analyze a gender gap in authorship of accepted abstracts to the American Academy of Neurology annual meetings in 2020 and 2021.
DESIGN/METHODS: This is a cross-sectional study evaluating the proportions of female first and senior abstract authors in 2020 and 2021. Abstracts were reviewed manually (
RESULTS: Accepted abstracts with female first and senior authors comprised 46%, 34% in 2020, and the same in 2021, without change. Female senior authors had a significantly higher proportion of female first authors than their male senior author counterparts. The analysis of subspecialties with more than 100 abstracts showed the lowest percentages of female senior authors was oncology (24.7%), sleep (25.5%), headache (28.7%), and cerebrovascular disease (29%) in 2020. Cerebrovascular disease (29%) and behavioral neurology (24.7%) had the lowest percentage of female senior authors in 2021. In the analysis of the origin of research, corporate-affiliated authors had the lowest percentages of female first (34 and 36%) and senior authors (22.6 and 27.6%).
CONCLUSION: The gender gap in neurology was reaffirmed in regards to female senior authorship overall and in subgroups of abstracts including cerebrovascular disease, headache, behavioral neurology, sleep, oncology, and corporate-affiliated research
Re-Evaluating the Gender Gap: A Cross-Sectional Analysis of Accepted American Academy of Neurology Annual Meeting Abstracts in 2020 and 2021
BACKGROUND AND OBJECTIVE: Prior studies reveal that invited speaker panels, editorial boards, authors of practice guidelines, and senior authors of published articles are disproportionately male in the neurology field. We aimed to analyze a gender gap in authorship of accepted abstracts to the American Academy of Neurology annual meetings in 2020 and 2021.
DESIGN/METHODS: This is a cross-sectional study evaluating the proportions of female first and senior abstract authors in 2020 and 2021. Abstracts were reviewed manually (
RESULTS: Accepted abstracts with female first and senior authors comprised 46%, 34% in 2020, and the same in 2021, without change. Female senior authors had a significantly higher proportion of female first authors than their male senior author counterparts. The analysis of subspecialties with more than 100 abstracts showed the lowest percentages of female senior authors was oncology (24.7%), sleep (25.5%), headache (28.7%), and cerebrovascular disease (29%) in 2020. Cerebrovascular disease (29%) and behavioral neurology (24.7%) had the lowest percentage of female senior authors in 2021. In the analysis of the origin of research, corporate-affiliated authors had the lowest percentages of female first (34 and 36%) and senior authors (22.6 and 27.6%).
CONCLUSION: The gender gap in neurology was reaffirmed in regards to female senior authorship overall and in subgroups of abstracts including cerebrovascular disease, headache, behavioral neurology, sleep, oncology, and corporate-affiliated research
Context-dependent Instruction Tuning for Dialogue Response Generation
Recent language models have achieved impressive performance in natural
language tasks by incorporating instructions with task input during
fine-tuning. Since all samples in the same natural language task can be
explained with the same task instructions, many instruction datasets only
provide a few instructions for the entire task, without considering the input
of each example in the task. However, this approach becomes ineffective in
complex multi-turn dialogue generation tasks, where the input varies highly
with each turn as the dialogue context changes, so that simple task
instructions cannot improve the generation performance. To address this
limitation, we introduce a context-based instruction fine-tuning framework for
each multi-turn dialogue which generates both responses and instructions based
on the previous context as input. During the evaluation, the model generates
instructions based on the previous context to self-guide the response. The
proposed framework produces comparable or even outstanding results compared to
the baselines by aligning instructions to the input during fine-tuning with the
instructions in quantitative evaluations on dialogue benchmark datasets with
reduced computation budget.Comment: Work in Progres
Effective Targeted Attacks for Adversarial Self-Supervised Learning
Recently, unsupervised adversarial training (AT) has been highlighted as a
means of achieving robustness in models without any label information. Previous
studies in unsupervised AT have mostly focused on implementing self-supervised
learning (SSL) frameworks, which maximize the instance-wise classification loss
to generate adversarial examples. However, we observe that simply maximizing
the self-supervised training loss with an untargeted adversarial attack often
results in generating ineffective adversaries that may not help improve the
robustness of the trained model, especially for non-contrastive SSL frameworks
without negative examples. To tackle this problem, we propose a novel positive
mining for targeted adversarial attack to generate effective adversaries for
adversarial SSL frameworks. Specifically, we introduce an algorithm that
selects the most confusing yet similar target example for a given instance
based on entropy and similarity, and subsequently perturbs the given instance
towards the selected target. Our method demonstrates significant enhancements
in robustness when applied to non-contrastive SSL frameworks, and less but
consistent robustness improvements with contrastive SSL frameworks, on the
benchmark datasets.Comment: NeurIPS 202
Learning Transferable Adversarial Robust Representations via Multi-view Consistency
Despite the success on few-shot learning problems, most meta-learned models
only focus on achieving good performance on clean examples and thus easily
break down when given adversarially perturbed samples. While some recent works
have shown that a combination of adversarial learning and meta-learning could
enhance the robustness of a meta-learner against adversarial attacks, they fail
to achieve generalizable adversarial robustness to unseen domains and tasks,
which is the ultimate goal of meta-learning. To address this challenge, we
propose a novel meta-adversarial multi-view representation learning framework
with dual encoders. Specifically, we introduce the discrepancy across the two
differently augmented samples of the same data instance by first updating the
encoder parameters with them and further imposing a novel label-free
adversarial attack to maximize their discrepancy. Then, we maximize the
consistency across the views to learn transferable robust representations
across domains and tasks. Through experimental validation on multiple
benchmarks, we demonstrate the effectiveness of our framework on few-shot
learning tasks from unseen domains, achieving over 10\% robust accuracy
improvements against previous adversarial meta-learning baselines.Comment: *Equal contribution (Author ordering determined by coin flip).
NeurIPS SafetyML workshop 2022, Under revie
Adaptation in pregnant women: a descriptive phenomenological study using Giorgi’s approach
Purpose This descriptive phenomenological study aimed to explore the lived experience and meaning of pregnant women’s adaptation. Methods Ten pregnant women from an ongoing Pregnant Couples' Cohort Study agreed to participate in this study. The data were collected through telephone in-depth interviews regarding what they experienced and felt about pregnancy adaptation. The qualitative data were analyzed using Giorgi’s method of descriptive phenomenology. Results Five core situation components were extracted from the raw data, along with 12 themes and 33 focal meanings. The five core situations were 1) first recognizing the pregnancy, 2) pregnancy-related changes, 3) the upcoming birth, 4) the postpartum period, and 5) parenting. The 12 themes were as follows: “anxiety, pressure, and embarrassment due to pregnancy,” “efforts to adapt to physical changes,” “efforts to adapt to the psychological difficulties of pregnancy,” “efforts to adapt to the financial burden and role changes caused by pregnancy,” “connecting with the fetus,” “adapting to a new marital relationship centering on the baby,” “the frustration of childbirth,” “fear of childbirth,” “postpartum care, need help with lactation planning,” “parenting beyond what I imagined” “dad’s willingness to participate in parenting,” and “career disconnect and consideration of workplace needs.” Conclusion We identified that pregnant women experience adaptation in physical, psychological, relational, and social aspects. The thematic clusters identified be used to develop nursing interventions to promote women's adaptation to pregnancy
Speech Enhancement for Virtual Meetings on Cellular Networks
We study speech enhancement using deep learning (DL) for virtual meetings on
cellular devices, where transmitted speech has background noise and
transmission loss that affects speech quality. Since the Deep Noise Suppression
(DNS) Challenge dataset does not contain practical disturbance, we collect a
transmitted DNS (t-DNS) dataset using Zoom Meetings over T-Mobile network. We
select two baseline models: Demucs and FullSubNet. The Demucs is an end-to-end
model that takes time-domain inputs and outputs time-domain denoised speech,
and the FullSubNet takes time-frequency-domain inputs and outputs the energy
ratio of the target speech in the inputs. The goal of this project is to
enhance the speech transmitted over the cellular networks using deep learning
models
- …