On Robustness and Bias Analysis of BERT-based Relation Extraction

Fine-tuning pre-trained models have achieved impressive performance on standard natural language processing benchmarks. However, the resultant model generalizability remains poorly understood. We do not know, for example, how excellent performance can lead to the perfection of generalization models. In this study, we analyze a fine-tuned BERT model from different perspectives using relation extraction. We also characterize the differences in generalization techniques according to our proposed improvements. From empirical experimentation, we find that BERT suffers a bottleneck in terms of robustness by way of randomizations, adversarial and counterfactual tests, and biases (i.e., selection and semantic). These findings highlight opportunities for future improvements. Our open-sourced testbed DiagnoseRE is available in \url{https://github.com/zjunlp/DiagnoseRE}.Comment: work in progres

Disentangled Contrastive Learning for Learning Robust Textual Representations

Although the self-supervised pre-training of transformer models has resulted in the revolutionizing of natural language processing (NLP) applications and the achievement of state-of-the-art results with regard to various benchmarks, this process is still vulnerable to small and imperceptible permutations originating from legitimate inputs. Intuitively, the representations should be similar in the feature space with subtle input permutations, while large variations occur with different meanings. This motivates us to investigate the learning of robust textual representation in a contrastive manner. However, it is non-trivial to obtain opposing semantic instances for textual samples. In this study, we propose a disentangled contrastive learning method that separately optimizes the uniformity and alignment of representations without negative sampling. Specifically, we introduce the concept of momentum representation consistency to align features and leverage power normalization while conforming the uniformity. Our experimental results for the NLP benchmarks demonstrate that our approach can obtain better results compared with the baselines, as well as achieve promising improvements with invariance tests and adversarial attacks. The code is available in https://github.com/zjunlp/DCL.Comment: Work in progres

Learning to Ask for Data-Efficient Event Argument Extraction

Event argument extraction (EAE) is an important task for information extraction to discover specific argument roles. In this study, we cast EAE as a question-based cloze task and empirically analyze fixed discrete token template performance. As generating human-annotated question templates is often time-consuming and labor-intensive, we further propose a novel approach called "Learning to Ask," which can learn optimized question templates for EAE without human annotations. Experiments using the ACE-2005 dataset demonstrate that our method based on optimized questions achieves state-of-the-art performance in both the few-shot and supervised settings.Comment: work in progres

An optimized short‐term steroid therapy for chronic drug‐induced liver injury: A prospective randomized clinical trial

Background and AimsThe use of corticosteroids in chronic drug-induced liver injury (DILI) is an important issue. Our previous randomized controlled trial showed that patients with chronic DILI benefited from a 48-week steroid stepwise reduction (SSR) regimen. However, it remains unclear whether a shorter course of therapy can achieve similar efficacy. In this study, we aimed to assess whether a 36-week SSR can achieve efficacy similar to that of 48-week SSR.MethodsA randomized open-label trial was performed. Eligible patients were randomly assigned to the 36- or 48-week (1:1) SSR group. Liver biopsies were performed at baseline and at the end of treatment. The primary outcome was the proportion of patients with relapse rate (RR). The secondary outcomes were improvement in liver histology and safety.ResultsOf the 90 participants enrolled, 84 (87.5%) completed the trial, and 62 patients (68.9%) were women. Hepatocellular damage was observed in 53.4% of the cohort. The RR was 7.1% in the 36-week SSR group but 4.8% in the 48-week SSR group, as determined by per-protocol set analysis (p = 1.000). Significant histological improvements in histological activity (93.1% vs. 92.9%, p = 1.000) and fibrosis (41.4% vs. 46.4%, p = .701) were observed in both the groups. Biochemical normalization time did not differ between the two groups. No severe adverse events were observed.ConclusionsBoth the 36- and 48-week SSR regimens demonstrated similar biochemical response and histological improvements with good safety, supporting 36-week SSR as a preferable therapeutic choice (ClinicalTrials.gov, NCT03266146)

New record of the genus Manipuria Jacoby (Chrysomelidae, Criocerinae) from China, with description of a new species

After a century since the erection of the genus Manipuria from India, its type species M. dohertyi Jacoby was discovered in Yunnan Province of China. A new Manipuria species, M. yuae sp. nov., is described from Tibet and Yunnan, China. The new species differs from M. dohertyi by its larger size, unicolored elytra, and absence of a tooth-like prolongation in front of the mandible. Additional data is provided for M. dohertyi based on new material from China

Snowfall Microphysics Characterized by PARSIVEL Disdrometer Observations in Beijing from 2020 to 2022

Accurate snowfall forecasting and quantitative snowfall estimation remain challenging due to the complexity and variability of snow microphysical properties. In this paper, the microphysical characteristics of snowfall in the Yanqing mountainous area of Beijing are investigated by using a Particle Size and Velocity (PARSIVEL) disdrometer. Results show that the high snowfall intensity process has large particle-size distribution (PSD) peak concentration, but the distribution of its spectrum width is much smaller than that of moderate or low snowfall intensity. When the snowfall intensity is high, the corresponding Dm value is smaller and the Nw value is larger. Comparison between the fitted μ−Λ relationship and the relationships of different locations show that there are regional differences. Based on dry snow samples, the Ze−SR relationship fitted in this paper is more consistent with the Ze−SR relationship of dry snow in Nanjing, China. The fitted ρs−Dm relationship of dry snow is close to the relationship in Pyeongchang, Republic of Korea, but the relationship of wet snow shows greatly difference. At last, the paper analyzes the statistics on velocity and diameter distribution of snow particles according to different snowfall intensities

Two‐stage voltage control strategy in distribution networks with coordinated multimode operation of PV inverters

Abstract To solve the problems of voltage violation and fast voltage fluctuation caused by photovoltaic (PV) grid connections in distribution networks, a two‐stage coordinated voltage control strategy for PV inverters with multimode coordinated operation in distribution networks (DNs) is proposed. It is based on the characteristics of PV inverters that can be regulated rapidly and continuously. First, the voltage control process is divided into two control stages according to the voltage change rate, and the corresponding voltage control models are established. Then, the operating modes of PV inverters are classified into three modes: economic prevention, economic control, and efficient control. Different operating modes are used to adapt to the two‐stage voltage control. Finally, a voltage fast control solution algorithm is proposed, which considers adaptive time delays and combines local measurement, prediction, and communication information to update control commands. The effectiveness and feasibility of the proposed voltage control strategy were demonstrated through simulation tests on a real 10‐kV line with a high PV penetration rate in the Anhui province of China. The results show that the proposed strategy can prevent voltage violation and sudden changes and can smooth out voltage fluctuation