Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attribute model decisions to individual input features with different saliency measures, but they fail to explain how these input features interact with each other to reach predictions. In this paper, we propose a self-attention attribution method to interpret the information interactions inside Transformer. We take BERT as an example to conduct extensive studies. Firstly, we apply self-attention attribution to identify the important attention heads, while others can be pruned with marginal performance degradation. Furthermore, we extract the most salient dependencies in each layer to construct an attribution tree, which reveals the hierarchical interactions inside Transformer. Finally, we show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.Comment: AAAI-202

Investigation of the topography-dependent current in conductive AFM and the calibration method

The topography and the electrical properties of materials are two crucial characteristics in determining their functionalities. Conductive atomic force microscopy (CAFM) is widely recognized for its ability to independently measure the topology and conductivity of the sample surface. The increasing trend towards miniaturization in electrical devices and sensors has led to an urgent demand for enhancing the accuracy of CAFM characterization. However, the sample's topography may affect the current measured by CAFM, leading to an inaccurate estimation of the sample's conductivity. Herein, we investigated the existence of topography-dependent current that originates from changes in capacitance between the probe and sample in CAFM testing. A linear correlation between the current and topography has been established using both experimental and theoretical methods. A calibration method based on this linear correlation has been proposed to eliminate the current error induced by the uneven surface of both insulators and conductors. This work will yield substantial advantages for research requiring high-precision CAFM testing.Comment: Corrected typo

A Pareto-based evolutionary algorithm using decomposition and truncation for dynamic multi-objective optimization

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link.Maintaining a balance between convergence and diversity of the population in the objective space has been widely recognized as the main challenge when solving problems with two or more conflicting objectives. This is added by another difficulty of tracking the Pareto optimal solutions set (POS) and/or the Pareto optimal front (POF) in dynamic scenarios. Confronting these two issues, this paper proposes a Pareto-based evolutionary algorithm using decomposition and truncation to address such dynamic multi-objective optimization problems (DMOPs). The proposed algorithm includes three contributions: a novel mating selection strategy, an efficient environmental selection technique and an effective dynamic response mechanism. The mating selection considers the decomposition-based method to select two promising mating parents with good diversity and convergence. The environmental selection presents a modified truncation method to preserve good diversity. The dynamic response mechanism is evoked to produce some solutions with good diversity and convergence whenever an environmental change is detected. In the experimental studies, a range of dynamic multi-objective benchmark problems with different characteristics were carried out to evaluate the performance of the proposed method. The experimental results demonstrate that the method is very competitive in terms of convergence and diversity, as well as in response speed to the changes, when compared with six other state-of-the-art methods

Educational degree differences in the association between work stress and depression among Chinese healthcare workers: Job satisfaction and sleep quality as the mediators

BackgroundDepressive status of medical personnel worldwide and especially in China is an important public health and social problem. There is a strong relationship between education and depression, but no studies have studied grouping healthcare workers (HCWs) with different educational degree to discuss whether there are differences in the factors that affect depression. This study aims to examine the role of job satisfaction and sleep quality in the relationship between work stress and depression among Chinese HCWs, and teste whether the mediation models are differed by the differences of educational degree.MethodsPatient Health Questionnaire-9 (PHQ-9) scale was used to test depression. Work stress was assessed using the Challenge-blocking stress scale (CBSS). Sleep quality was assessed using the Pittsburgh Sleep Quality Index (PSQI). HCWs’ satisfaction with their current work was assessed using the Job Satisfaction Index (JSI). The representative sample of HCWs was chosen using a multi-stage stratified cluster random sampling procedure and 844 HCWs were utilized to the statistical analysis of the study.ResultsIn the overall sample, sleep quality could mediate the relationship between work stress and depression in healthcare workers (p < 0.001, CMIN/DF = 3.816, GFI = 0.911, AGFI = 0.886, IFI = 0.943, TLI = 0.933, CFI = 0.942, RMSEA = 0.058, SRMR = 0.055, AIC = 1039.144), and the mediating effect accounted for 36.5%. After grouping educational qualifications, the model with sleep quality and job satisfaction as mediating variables reported a better fit in the group with low educational qualifications. The intermediary effect accounted for 50.6 and 4.43%, respectively. The highly educated group only has sleep quality as an intermediary variable in the structural model, and the mediating effect accounted for 75.4% (p < 0.001, CMIN/DF = 2.596, GFI = 0.887, AGFI = 0.857, IFI = 0.937, TLI = 0.926, CFI = 0.937, RMSEA = 0.044, SRMR = 0.056, AIC = 1481.322).ConclusionIn the overall sample, sleep quality could mediate the relationship between work stress and depression in HCWs. Among HCWs with technical secondary school education and below, job satisfaction can mediate the positive relationship between work stress and depression, while this mediating effect is not significant among HCWs with college degree and above

PSTPIP2 Inhibits the Inflammatory Response and Proliferation of Fibroblast-Like Synoviocytes in vitro

Rheumatoid arthritis (RA) is a chronic autoimmune inflammatory disease and its pathogenesis remains unclear. Fibroblast-like synoviocytes (FLSs) play an important role in the pathogenesis of RA. Proline-serine-threonine phosphatase interacting protein 2 (PSTPIP2) is an adaptor protein, which is associated with auto-inflammatory disease. In this study, we selected adjuvant-induced arthritis (AIA) as animal model to study the role of PSTPIP2 in FLSs. We found that the expression of PSTPIP2 was significantly down-regulated in synovial tissues and FLSs of AIA rat compared with normal group. And overexpression of PSTPIP2 could inhibit the proliferation and inflammatory response of FLSs. Moreover, the proliferation and inflammatory response of FLSs were promoted with PSTPIP2 silencing treatment. In terms of mechanism, we found that the expression of PSTPIP2 was closely related to NF-κB signaling pathway. Overall, our results suggested that PSTPIP2 inhibits the proliferation and inflammatory response of FLSs, which might be closely related to NF-κB signaling pathway

Short- and long-term effects of manganese, zinc and copper ions on nitrogen removal in nitritation-anammox process

This study provided a deep insight into the impacts of trace elements (Mn2+, Zn2+ and Cu2+) on nitritation-anammox process. For short-term exposure, all the three elements could improve the nitrogen removal rate (NRR) and the optimal concentrations were 2.0 mg/L, 2.0 mg/L and 0.5 mg/L for Mn2+, Zn2+ and Cu2+, respectively. Accordingly, the NRR were enhanced 54.62%, 45.93% and 44.09%. The long-term experiments were carried out in lab-scale sequencing batch reactors. The surprising results showed that only Mn2+ addition could enhance the long-term nitritation-anammox process, and the NRR increased from 0.35 ± 0.01 kg N/m3/d (control, no extra trace element addition) to 0.49 ± 0.03 kg N/m3/d. Vice versa, the amendment of Zn2+ reduced the NRR to 0.28 ± 0.02 kg N/m3/d, and Cu2+ had no significant effect on the NRR (0.36 ± 0.01 kg N/m3/d). From the analysis of microbial community structure, it was explained by the increasing abundance of anaerobic ammonium oxidizing bacteria (AnAOB) only in Mn2+ treatment, whereas Zn2+ predominantly promoted ammonium oxidizing bacteria (AOB). Additionally, the majority of Mn2+ was identified inside AnAOB cells, and Zn2+ and Cu2+ were mainly located in AOB. Our results indicated the synergistic effects of trace elements on nitritation-anammox, both short-term encouraging activities of AnAOB and long-term altering microbial community structure. This work implies the importance of trace elements addition in nitritation-anammox process

A Synthetic Plasmid Toolkit for Shewanella oneidensis MR-1

Shewanella oneidensis MR-1 is a platform microorganism for understanding extracellular electron transfer (EET) with a fully sequenced and annotated genome. In comparison to other model microorganisms such as Escherichia coli, the available plasmid parts (such as promoters and replicons) are not sufficient to conveniently and quickly fine-tune the expression of multiple genes in S. oneidensis MR-1. Here, we constructed and characterized a plasmid toolkit that contains a set of expression vectors with a combination of promoters, replicons, antibiotic resistance genes, and an RK2 origin of transfer (oriT) cassette, in which each element can be easily changed by fixed restriction enzyme sites. The expression cassette is also compatible with BioBrick synthetic biology standards. Using green fluorescent protein (GFP) as a reporter, we tested and quantified the strength of promoters. The copy number of different replicons was also measured by real-time quantitative PCR. We further transformed two compatible plasmids with different antibiotic resistance genes into the recombinant S. oneidensis MR-1, enabling control over the expression of two different fluorescent proteins. This plasmid toolkit was further used for overexpression of the MtrCAB porin-c-type cytochrome complex in the S. oneidensis ΔmtrA strain. Tungsten trioxide (WO3) reduction and microbial fuel cell (MFC) assays revealed that the EET efficiency was improved most significantly when MtrCAB was expressed at a moderate level, thus demonstrating the utility of the plasmid toolkit in the EET regulation in S. oneidensis. The plasmid toolkit developed in this study is useful for rapid and convenient fine-tuning of gene expression and enhances the ability to genetically manipulate S. oneidensis MR-1

Genome-wide analysis of FRF gene family and functional identification of HvFRF9 under drought stress in barley

FHY3 and its homologous protein FAR1 are the founding members of FRS family. They exhibited diverse and powerful physiological functions during evolution, and participated in the response to multiple abiotic stresses. FRF genes are considered to be truncated FRS family proteins. They competed with FRS for DNA binding sites to regulate gene expression. However, only few studies are available on FRF genes in plants participating in the regulation of abiotic stress. With wide adaptability and high stress-resistance, barley is an excellent candidate for the identification of stress-resistance-related genes. In this study, 22 HvFRFs were detected in barley using bioinformatic analysis from whole genome. According to evolution and conserved motif analysis, the 22 HvFRFs could be divided into subfamilies I and II. Most promoters of subfamily I members contained abscisic acid and methyl jasmonate response elements; however, a large number promoters of subfamily II contained gibberellin and salicylic acid response elements. HvFRF9, one of the members of subfamily II, exhibited a expression advantage in different tissues, and it was most significantly upregulated under drought stress. In-situ PCR revealed that HvFRF9 is mainly expressed in the root epidermal cells, as well as xylem and phloem of roots and leaves, indicating that HvFRF9 may be related to absorption and transportation of water and nutrients. The results of subcellular localization indicated that HvFRF9 was mainly expressed in the nuclei of tobacco epidermal cells and protoplast of arabidopsis. Further, transgenic arabidopsis plants with HvFRF9 overexpression were generated to verify the role of HvFRF9 in drought resistance. Under drought stress, leaf chlorosis and wilting, MDA and O2− contents were significantly lower, meanwhile, fresh weight, root length, PRO content, and SOD, CAT and POD activities were significantly higher in HvFRF9-overexpressing arabidopsis plants than in wild-type plants. Therefore, overexpression of HvFRF9 could significantly enhance the drought resistance in arabidopsis. These results suggested that HvFRF9 may play a key role in drought resistance in barley by increasing the absorption and transportation of water and the activity of antioxidant enzymes. This study provided a theoretical basis for drought resistance in barley and provided new genes for drought resistance breeding