Language Detoxification with Attribute-Discriminative Latent Space

Transformer-based Language Models (LMs) have achieved impressive results on natural language understanding tasks, but they can also generate toxic text such as insults, threats, and profanity, limiting their real-world applications. To overcome this issue, a few text generation approaches aim to detoxify toxic texts using additional LMs or perturbations. However, previous methods require excessive memory, computations, and time which are serious bottlenecks in their real-world application. To address such limitations, we propose an effective yet efficient method for language detoxification using an attribute-discriminative latent space. Specifically, we project the latent space of an original Transformer LM onto a discriminative latent space that well-separates texts by their attributes using a projection block and an attribute discriminator. This allows the LM to control the text generation to be non-toxic with minimal memory and computation overhead. We validate our model, Attribute-Discriminative Language Model (ADLM) on detoxified language and dialogue generation tasks, on which our method significantly outperforms baselines both in performance and efficiency.Comment: ACL 2023; *Equal contribution. Author ordering determined by coin fli

Re-Evaluating the Gender Gap: a Cross-Sectional analysis of accepted american academy of Neurology annual Meeting abstracts in 2020 and 2021

BACKGROUND AND OBJECTIVE: Prior studies reveal that invited speaker panels, editorial boards, authors of practice guidelines, and senior authors of published articles are disproportionately male in the neurology field. We aimed to analyze a gender gap in authorship of accepted abstracts to the American Academy of Neurology annual meetings in 2020 and 2021. DESIGN/METHODS: This is a cross-sectional study evaluating the proportions of female first and senior abstract authors in 2020 and 2021. Abstracts were reviewed manually ( RESULTS: Accepted abstracts with female first and senior authors comprised 46%, 34% in 2020, and the same in 2021, without change. Female senior authors had a significantly higher proportion of female first authors than their male senior author counterparts. The analysis of subspecialties with more than 100 abstracts showed the lowest percentages of female senior authors was oncology (24.7%), sleep (25.5%), headache (28.7%), and cerebrovascular disease (29%) in 2020. Cerebrovascular disease (29%) and behavioral neurology (24.7%) had the lowest percentage of female senior authors in 2021. In the analysis of the origin of research, corporate-affiliated authors had the lowest percentages of female first (34 and 36%) and senior authors (22.6 and 27.6%). CONCLUSION: The gender gap in neurology was reaffirmed in regards to female senior authorship overall and in subgroups of abstracts including cerebrovascular disease, headache, behavioral neurology, sleep, oncology, and corporate-affiliated research

Context-dependent Instruction Tuning for Dialogue Response Generation

Recent language models have achieved impressive performance in natural language tasks by incorporating instructions with task input during fine-tuning. Since all samples in the same natural language task can be explained with the same task instructions, many instruction datasets only provide a few instructions for the entire task, without considering the input of each example in the task. However, this approach becomes ineffective in complex multi-turn dialogue generation tasks, where the input varies highly with each turn as the dialogue context changes, so that simple task instructions cannot improve the generation performance. To address this limitation, we introduce a context-based instruction fine-tuning framework for each multi-turn dialogue which generates both responses and instructions based on the previous context as input. During the evaluation, the model generates instructions based on the previous context to self-guide the response. The proposed framework produces comparable or even outstanding results compared to the baselines by aligning instructions to the input during fine-tuning with the instructions in quantitative evaluations on dialogue benchmark datasets with reduced computation budget.Comment: Work in Progres

Effective Targeted Attacks for Adversarial Self-Supervised Learning

Recently, unsupervised adversarial training (AT) has been highlighted as a means of achieving robustness in models without any label information. Previous studies in unsupervised AT have mostly focused on implementing self-supervised learning (SSL) frameworks, which maximize the instance-wise classification loss to generate adversarial examples. However, we observe that simply maximizing the self-supervised training loss with an untargeted adversarial attack often results in generating ineffective adversaries that may not help improve the robustness of the trained model, especially for non-contrastive SSL frameworks without negative examples. To tackle this problem, we propose a novel positive mining for targeted adversarial attack to generate effective adversaries for adversarial SSL frameworks. Specifically, we introduce an algorithm that selects the most confusing yet similar target example for a given instance based on entropy and similarity, and subsequently perturbs the given instance towards the selected target. Our method demonstrates significant enhancements in robustness when applied to non-contrastive SSL frameworks, and less but consistent robustness improvements with contrastive SSL frameworks, on the benchmark datasets.Comment: NeurIPS 202

Learning Transferable Adversarial Robust Representations via Multi-view Consistency

Despite the success on few-shot learning problems, most meta-learned models only focus on achieving good performance on clean examples and thus easily break down when given adversarially perturbed samples. While some recent works have shown that a combination of adversarial learning and meta-learning could enhance the robustness of a meta-learner against adversarial attacks, they fail to achieve generalizable adversarial robustness to unseen domains and tasks, which is the ultimate goal of meta-learning. To address this challenge, we propose a novel meta-adversarial multi-view representation learning framework with dual encoders. Specifically, we introduce the discrepancy across the two differently augmented samples of the same data instance by first updating the encoder parameters with them and further imposing a novel label-free adversarial attack to maximize their discrepancy. Then, we maximize the consistency across the views to learn transferable robust representations across domains and tasks. Through experimental validation on multiple benchmarks, we demonstrate the effectiveness of our framework on few-shot learning tasks from unseen domains, achieving over 10\% robust accuracy improvements against previous adversarial meta-learning baselines.Comment: *Equal contribution (Author ordering determined by coin flip). NeurIPS SafetyML workshop 2022, Under revie

Adaptation in pregnant women: a descriptive phenomenological study using Giorgi’s approach

Purpose This descriptive phenomenological study aimed to explore the lived experience and meaning of pregnant women’s adaptation. Methods Ten pregnant women from an ongoing Pregnant Couples' Cohort Study agreed to participate in this study. The data were collected through telephone in-depth interviews regarding what they experienced and felt about pregnancy adaptation. The qualitative data were analyzed using Giorgi’s method of descriptive phenomenology. Results Five core situation components were extracted from the raw data, along with 12 themes and 33 focal meanings. The five core situations were 1) first recognizing the pregnancy, 2) pregnancy-related changes, 3) the upcoming birth, 4) the postpartum period, and 5) parenting. The 12 themes were as follows: “anxiety, pressure, and embarrassment due to pregnancy,” “efforts to adapt to physical changes,” “efforts to adapt to the psychological difficulties of pregnancy,” “efforts to adapt to the financial burden and role changes caused by pregnancy,” “connecting with the fetus,” “adapting to a new marital relationship centering on the baby,” “the frustration of childbirth,” “fear of childbirth,” “postpartum care, need help with lactation planning,” “parenting beyond what I imagined” “dad’s willingness to participate in parenting,” and “career disconnect and consideration of workplace needs.” Conclusion We identified that pregnant women experience adaptation in physical, psychological, relational, and social aspects. The thematic clusters identified be used to develop nursing interventions to promote women's adaptation to pregnancy

Speech Enhancement for Virtual Meetings on Cellular Networks

We study speech enhancement using deep learning (DL) for virtual meetings on cellular devices, where transmitted speech has background noise and transmission loss that affects speech quality. Since the Deep Noise Suppression (DNS) Challenge dataset does not contain practical disturbance, we collect a transmitted DNS (t-DNS) dataset using Zoom Meetings over T-Mobile network. We select two baseline models: Demucs and FullSubNet. The Demucs is an end-to-end model that takes time-domain inputs and outputs time-domain denoised speech, and the FullSubNet takes time-frequency-domain inputs and outputs the energy ratio of the target speech in the inputs. The goal of this project is to enhance the speech transmitted over the cellular networks using deep learning models

Development and application of a couple-centered antenatal education program in Korea

Purpose This study was conducted to develop a couple-centered antenatal education program and to test the program’s feasibility. Methods With a preliminary-experimental study design, 33 pregnant couples who were expecting their first child participated in this study. The program consisted of four sessions (1 hour/session/week) of education and counseling. Data were collected before and after the intervention from September 2018 to April 2019 at a women’s hospital in Daejeon, Korea, with demographic data forms, the Edinburgh Postnatal Depression Scale, Perceived Stress Scale, Maternal–Fetal Attachment Scale, Korean Newborn Care Confidence Scale, Wijma Delivery Expectancy/Experience Questionnaire, and Dyadic Adjustment Scale-10. Results The pregnant women and their husbands were on average 32.30±3.10 and 33.21±6.25 years old, respectively. The mean marriage duration was 2.34±1.63 years, the gestational age was 31.30±2.66 weeks, and 78.8% of the couples had a planned pregnancy. After the program, both the pregnant women and their husbands showed significant improvements in attachment to the fetus and confidence in providing infant care. Prenatal depression, prenatal stress, and fear of childbirth in pregnant women significantly decreased after completing the program. However, the dyadic adjustment score did not change significantly either in the pregnant women or their husbands. Conclusion A couple-centered antenatal education program seems to be effective for couples adjusting to parenthood, but further studies should explore ways to have a positive impact on couples’ relationships