Diet Code Is Healthy: Simplifying Programs for Pre-trained Models of Code

Pre-trained code representation models such as CodeBERT have demonstrated superior performance in a variety of software engineering tasks, yet they are often heavy in complexity, quadratically with the length of the input sequence. Our empirical analysis of CodeBERT's attention reveals that CodeBERT pays more attention to certain types of tokens and statements such as keywords and data-relevant statements. Based on these findings, we propose DietCode, which aims at lightweight leverage of large pre-trained models for source code. DietCode simplifies the input program of CodeBERT with three strategies, namely, word dropout, frequency filtering, and an attention-based strategy which selects statements and tokens that receive the most attention weights during pre-training. Hence, it gives a substantial reduction in the computational cost without hampering the model performance. Experimental results on two downstream tasks show that DietCodeBERT provides comparable results to CodeBERT with 40% less computational cost in fine-tuning and testing.Comment: Accepted to be published in ESEC/FSE 202

Cost-benefit analysis of integrated energy system planning considering demand response

The power-gas-coupling can realize the cascade utilization of energy in the integrated energy system, which is conducive to improving the utilization of energy and reducing pollution gases emissions. With the installation of smart metering, two-way communication between suppliers and consumers is feasible, which enables the implementation of demand response. A generic optimal planning model is proposed to assess the economic and environmental benefits of the capacity allocation of the grid-connected integrated energy system considering both price-based demand response and incentive-based demand response respectively. The optimal planning problem is formulated as a mixed-integer linear programming model with the objective to minimize the total annual cost. The results from three configuration modes are compared in the case study, which illustrate the economic and environmental benefits from demand response. In addition, the impact of the sales capacity constraint on the grid and the fluctuation of electricity and gas prices on the planning of the integrated energy system are also extensively studied considering demand response

Endoplasmic Reticulum Stress-Mediated Apoptosis Involved in Indirect Recognition Pathway Blockade Induces Long-Term Heart Allograft Survival

Implementation of dendritic cell- (DC-) based therapies in organ transplantation can reduce dependency on nonspecific immunosuppression. Despite extensive research, mechanisms of equipped DCs inducing transplant tolerance remain incomplete. Here, we applied RNA interference technique to inhibit CD80 and CD86 expression in host bone marrow-derived DCs. This approach could specifically and effectively knock down CD80 and CD86 expression. T cells primed by these DCs inhibited allogeneic responses. Administration of recipient DCs loaded with alloantigen after CD80 and CD86 blockade prolonged cardiac allograft survival. We also found a higher percentage of apoptotic T cells in lymph tissues and grafts than that detected in control group. In addition, these T cells expressed high expression of GRP78 than controls, indicating activation of unfolded protein responses. Upregulation of CHOP expression among these cells suggested that the endoplasmic reticulum stress (ERS) response switched to a proapoptotic response. Our results indicated that ERS-induced apoptosis may be involved in allogeneic T-cell apoptosis, and the ERS-mediated apoptosis pathway may be a novel target in clinical prevention and therapy of allograft rejection

InfeRE: Step-by-Step Regex Generation via Chain of Inference

Automatically generating regular expressions (abbrev. regexes) from natural language description (NL2RE) has been an emerging research area. Prior studies treat regex as a linear sequence of tokens and generate the final expressions autoregressively in a single pass. They did not take into account the step-by-step internal text-matching processes behind the final results. This significantly hinders the efficacy and interpretability of regex generation by neural language models. In this paper, we propose a new paradigm called InfeRE, which decomposes the generation of regexes into chains of step-by-step inference. To enhance the robustness, we introduce a self-consistency decoding mechanism that ensembles multiple outputs sampled from different models. We evaluate InfeRE on two publicly available datasets, NL-RX-Turk and KB13, and compare the results with state-of-the-art approaches and the popular tree-based generation approach TRANX. Experimental results show that InfeRE substantially outperforms previous baselines, yielding 16.3% and 14.7% improvement in DFA@5 accuracy on two datasets, respectively. Particularly, InfeRE outperforms the popular tree-based generation approach by 18.1% and 11.3% on both datasets, respectively, in terms of DFA@5 accuracy.Comment: This paper has been accepted by ASE'2