Effective Unsupervised Domain Adaptation with Adversarially Trained Language Models

Recent work has shown the importance of adaptation of broad-coverage contextualised embedding models on the domain of the target task of interest. Current self-supervised adaptation methods are simplistic, as the training signal comes from a small percentage of \emph{randomly} masked-out tokens. In this paper, we show that careful masking strategies can bridge the knowledge gap of masked language models (MLMs) about the domains more effectively by allocating self-supervision where it is needed. Furthermore, we propose an effective training strategy by adversarially masking out those tokens which are harder to reconstruct by the underlying MLM. The adversarial objective leads to a challenging combinatorial optimisation problem over \emph{subsets} of tokens, which we tackle efficiently through relaxation to a variational lowerbound and dynamic programming. On six unsupervised domain adaptation tasks involving named entity recognition, our method strongly outperforms the random masking strategy and achieves up to +1.64 F1 score improvements.Comment: EMNLP202

On the modelling and impact of negative edges in graph convolutional networks for node classification

Signed graphs are important data structures to simultaneously express positive and negative relationships. Their application ranges from structural health monitoring to financial models, where the meaning and properties of negative relationships can play a significant role. In this paper, we provide a comprehensive examination of existing approaches for the integration of signed edges into the Graph Convolutional Network (GCN) framework for node classification. Here, we use a combination of theoretical and empirical analysis to gain a deeper understanding of the strengths and limitations of different mechanisms and to identify areas for possible improvement. We compare six different approaches to the integration of negative link information within the framework of the simple GCN. In particular, we analyze sensitivity towards feature noise, negative edge noise and positive edge noise, as well as robustness towards feature scaling and translation, explaining the results obtained on the basis of individual model assumptions and biases. Our findings highlight the importance of capturing the meaning of negative links in a given domain context, and appropriately reflecting it in the choice of GCN model. Our code is available at https://github.com/dinhtrang24/Signed-GCN

On a Standard Model Extension with Vector-like Fermions and Abelian Symmetry

We investigate an extension of the standard model with vector-like fermions and an extra Abelian gauge symmetry. The particle mass spectrum is calculated explicitly. The Lagrangian terms for all the gauge interactions of leptons and quarks in the model are derived. We observe that while there is no new mixing in the lepton sector, the quark mixing plays an important role in the magnitudes of gauge interactions for quarks, particularly the interactions with massive $W$, $Z$ and $Z'$ bosons. We calculate the contributions of the new vector-like leptons and quarks to the Peskin-Takeuchi parameters as well as the $\rho$ parameter of the electroweak precision tests, and show that the model is realistic

Systematic Assessment of Factual Knowledge in Large Language Models

Previous studies have relied on existing question-answering benchmarks to evaluate the knowledge stored in large language models (LLMs). However, this approach has limitations regarding factual knowledge coverage, as it mostly focuses on generic domains which may overlap with the pretraining data. This paper proposes a framework to systematically assess the factual knowledge of LLMs by leveraging knowledge graphs (KGs). Our framework automatically generates a set of questions and expected answers from the facts stored in a given KG, and then evaluates the accuracy of LLMs in answering these questions. We systematically evaluate the state-of-the-art LLMs with KGs in generic and specific domains. The experiment shows that ChatGPT is consistently the top performer across all domains. We also find that LLMs performance depends on the instruction finetuning, domain and question complexity and is prone to adversarial context.Comment: Accepted by EMNLP 2023 Finding

Neonatal Outcomes in the Surgical Management of Placenta Accreta Spectrum Disorders: A Retrospective Single-Center Observational Study From 468 Vietnamese Pregnancies Beyond 28 Weeks of Gestation

BACKGROUND: Placenta accreta spectrum disorders (PASDs) increase the mortality rate for mothers and newborns over a decade. Thus, the purpose of the study is to evaluate the neonatal outcomes in emergency cesarean section (CS) and planned surgery as well as in Cesarean hysterectomy and the modified one-step conservative uterine surgery (MOSCUS). The secondary aim is to reveal the factors relating to poor neonatal outcomes. METHODS: This was a single-center retrospective study conducted between 2019 and 2020 at Tu Du Hospital, in the southern region of Vietnam. A total of 497 pregnant women involved in PASDs beyond 28 weeks of gestation were enrolled. The clinical outcomes concerning gestational age, birth weight, APGAR score, neonatal intervention, neonatal intensive care unit (NICU) admission, and NICU length of stay (LOS) were compared between emergency and planned surgery, between the Cesarean hysterectomy and the MOSCUS. The univariate and multivariable logistic regression were used to assess the adverse neonatal outcomes. RESULTS: Among 468 intraoperatively diagnosed PASD cases who underwent CS under general anesthesia, neonatal outcomes in the emergency CS (n = 65) were significantly poorer than in planned delivery (n = 403). Emergency CS increased the odds ratio (OR) for earlier gestational age, lower birthweight, lower APGAR score at 5 min, higher rate of neonatal intervention, NICU admission, and longer NICU LOS ≥ 7 days with OR, 95% confidence interval (CI) were 10.743 (5.675-20.338), 3.823 (2.197-6.651), 5.215 (2.277-11.942), 2.256 (1.318-3.861), 2.177 (1.262-3.756), 3.613 (2.052-6.363), and 2.298 (1.140-4.630), respectively, p \u3c 0.05. Conversely, there was no statistically significant difference between the neonatal outcomes in Cesarean hysterectomy (n = 79) and the MOSCUS method (n = 217). Using the multivariable logistic regression, factors independently associated with the 5-min-APGAR score of less than 7 points were time duration from the skin incision to fetal delivery (min) and gestational age (week). One minute-decreased time duration from skin incision to fetal delivery contributed to reduce the risk of adverse neonatal outcome by 2.2% with adjusted OR, 95% CI: 0.978 (0.962-0.993), p = 0.006. Meanwhile, one week-decreased gestational age increased approximately two fold odds of the adverse neonatal outcome with adjusted OR, 95% CI: 1.983 (1.600-2.456), p \u3c 0.0001. CONCLUSIONS: Among pregnancies with PASDs, the neonatal outcomes are worse in the emergency group compared to planned group of cesarean section. Additionally, the neonatal comorbidities in the conservative surgery using the MOSCUS method are similar to Cesarean hysterectomy. Time duration from the skin incision to fetal delivery and gestational age may be considered in PASD surgery. Further data is required to strengthen these findings

Expression, purification and evaluation of recombinant L-asparaginase inmehthylotrophic yeast Pichia pastoris: Expression, purification and evaluation of recombinant L-asparaginase in mehthylotrophic yeast Pichia pastoris: Research article

L-asparaginase (EC 3.5.1.1), a therapeutic enzyme used in the treatment of childhood acute lymphoblastic leukemia (ALL). Hence, the goal of this work is study the expression and evaluation of hydrolysis activity of native sequence (X12746) encoding for L-asparaginase from Erwinia chrysanthemi NCPBB1125 in the popular expression system Pichia pastoris. The sequence of asn encoded for mature protein was expressed in P. pastoris SMD1168 and X33. SDS-PAGE analysis showed recombinant L-asparaginase was secreted efficiently. Stable and high hydrolysis activity of extracellular L-asparaginase in P. pastoris SMD1168 making it a potential candidate to produce recombinant protein. After purification, a specific band whose appearance approximately 45 kDa indicating the glycosylated protein with specific activity by 6.251 Umg-1 and about 3 folds purifications.L-asparaginase (EC 3.5.1.1), một loại enzyme được sử dụng trong điều trị bệng ung thư bạch cầu mãn tính ở trẻ em. Mục tiêu của nghiên cứu này là biểu hiện và đánh giá hoạt tính thủy phân của L-asparaginase mã hóa bởi đoạn gene (X12746) tương ứng từ Erwinia chrysanthemi NCPBB1125 được biểu hiện trong nấm men Pichia pastoris. Gene đã được cắt signal peptide và biểu hiện trong P. pastoris SMD1168 and X33. Qua phân tích kết quả điện di SDS-PAGE của môi trường sau lên men, L-asparaginase tái tổ hợp được tìm thấy trong dịch ngoại bào của P. pastoris. Với khả năng sản xuất protein có hoạt tính cao hơn so với chủng P. pastoris X33, SMD1168 được lựa chọn để biểu hiện L-asparaginase tái tổ hợp. Sau khi tinh sạch, sự xuất hiện của một băng có kích khối lượng phân tử xấp xỉ 45 kDa trên điện di SDS-PAGE cho thấy protein tái tổ hợp đã bị glycosyl hóa với hoạt tính riêng 6.251 Umg-1 và đạt độ sạch 3.471 lần

ENGLISH-MAJORED STUDENTS’ LISTENING DIFFICULTIES AND USE OF STRATEGIES AT MIEN DONG UNIVERSITY OF TECHNOLOGY, VIETNAM

Through the years, difficulties in English listening and listening strategies have been conducted by many researchers. Most of studies have pointed out the common difficulties and strategies which students often have. Focusing on the same subject with a new perspective, this research aimed at understanding English majors’ difficulties in listening and use of listening strategies. A total of ninety eight freshmen English-majored students at Mien Dong University took part in answering the questionnaire and five students answered the semi-structure interview questions. The data gained from the questionnaire were analyzed by SPSS version 20.0 in terms of descriptive statistic. The findings revealed that English-majored students had many difficulties in listening such as the difficulties related to the listener, the content of the dialogue, the speaker, the physical setting and the linguistic factors. In terms of listening strategies, the finding showed that the cognitive listening strategies were used more frequently than metacognitive and socio-affective strategies. Based on the findings, some implications were made to contribute to the administrations, teachers and students at Mien Dong University of Technology.  Article visualizations