Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translation

Benefiting from the sequence-level knowledge distillation, the Non-Autoregressive Transformer (NAT) achieves great success in neural machine translation tasks. However, existing knowledge distillation has side effects, such as propagating errors from the teacher to NAT students, which may limit further improvements of NAT models and are rarely discussed in existing research. In this paper, we introduce selective knowledge distillation by introducing an NAT evaluator to select NAT-friendly targets that are of high quality and easy to learn. In addition, we introduce a simple yet effective progressive distillation method to boost NAT performance. Experiment results on multiple WMT language directions and several representative NAT models show that our approach can realize a flexible trade-off between the quality and complexity of training data for NAT models, achieving strong performances. Further analysis shows that distilling only 5% of the raw translations can help an NAT outperform its counterpart trained on raw data by about 2.4 BLEU

Latent Opinions Transfer Network for Target-Oriented Opinion Words Extraction

Target-oriented opinion words extraction (TOWE) is a new subtask of ABSA, which aims to extract the corresponding opinion words for a given opinion target in a sentence. Recently, neural network methods have been applied to this task and achieve promising results. However, the difficulty of annotation causes the datasets of TOWE to be insufficient, which heavily limits the performance of neural models. By contrast, abundant review sentiment classification data are easily available at online review sites. These reviews contain substantial latent opinions information and semantic patterns. In this paper, we propose a novel model to transfer these opinions knowledge from resource-rich review sentiment classification datasets to low-resource task TOWE. To address the challenges in the transfer process, we design an effective transformation method to obtain latent opinions, then integrate them into TOWE. Extensive experimental results show that our model achieves better performance compared to other state-of-the-art methods and significantly outperforms the base model without transferring opinions knowledge. Further analysis validates the effectiveness of our model.Comment: Accepted by the 34th AAAI Conference on Artificial Intelligence (AAAI 2020

BLEURT Has Universal Translations: An Analysis of Automatic Metrics by Minimum Risk Training

Automatic metrics play a crucial role in machine translation. Despite the widespread use of n-gram-based metrics, there has been a recent surge in the development of pre-trained model-based metrics that focus on measuring sentence semantics. However, these neural metrics, while achieving higher correlations with human evaluations, are often considered to be black boxes with potential biases that are difficult to detect. In this study, we systematically analyze and compare various mainstream and cutting-edge automatic metrics from the perspective of their guidance for training machine translation systems. Through Minimum Risk Training (MRT), we find that certain metrics exhibit robustness defects, such as the presence of universal adversarial translations in BLEURT and BARTScore. In-depth analysis suggests two main causes of these robustness deficits: distribution biases in the training datasets, and the tendency of the metric paradigm. By incorporating token-level constraints, we enhance the robustness of evaluation metrics, which in turn leads to an improvement in the performance of machine translation systems. Codes are available at \url{https://github.com/powerpuffpomelo/fairseq_mrt}.Comment: Accepted to ACL 2023 main conferenc

Research on the mechanism of neutral-point voltage fluctuation and capacitor voltage balancing control strategy of three-phase three-level T-type inverter

In order to solve the neutral-point voltage fluctuation problem of three-phase threelevel T-type inverters (TPTLTIs), the unbalance characteristics of capacitor voltages under different switching states and the mechanism of neutral-point voltage fluctuation are revealed. Based on the mathematical model of a TPTLTI, a feed-forward voltage balancing control strategy of DC-link capacitor voltages error is proposed. The strategy generates a DC bias voltage using a capacitor voltage loop with a proportional integral (PI) controller. The proposed strategy can suppress the neutral-point voltage fluctuation effectively and improve the quality of output currents. The correctness of the theoretical analysis is verified through simulations. An experimental prototype of a TPTLTI based on Digital Signal Processor (DSP) is built. The feasibility and effectiveness of the proposed strategy is verified through experiment. The results from simulations and experiment match very well

Study on dynamic strength and liquefaction mechanism of silt soil in Castor earthquake prone areas under different consolidation ratios

Under the Castor earthquake, there is a risk of liquefaction instability of saturated tailings, and the evolution of dynamic pore pressure can indirectly reflect its instability process. Before applying dynamic loads, the static stress state of soil is one of the main factors affecting the development of soil dynamic strength and dynamic pore pressure, and there are significant differences in soil dynamic strength under different consolidation ratios. This paper conducted dynamic triaxial tests on saturated tailings silt with different consolidation ratios, and analyzed the dynamic strength variation and liquefaction mechanism of the samples using the discrete element method (PFC3D). The results showed that 1) as the Kc′ gradually increased, and there was a critical consolidation ratio Kc′ during the development of the dynamic strength of the sample. The specific value of Kc′ was related to the properties and stress state of saturated sand. The Kc′ in this research was about 1.9. When Kc < 1.9, dynamic strength was increased with the increase in Kc; when Kc > 1.9, dynamic strength was decreased with the Kc. 2) Under the impact of cyclic load, when samples were normally consolidated (Kc =1), the pore water pressure would tend to be equal to the confining pressure to cause soil liquefaction. In the case of eccentric consolidation (Kc > 1), the pore water pressure would be less than the confining pressure, thus, the soil liquefaction would not be induced, and the pore pressure value would decrease with the increase of consolidation ratio. This paper provides engineering guidance value for the study of dynamic strength and liquefaction mechanism of tailings sand and silt in Castor earthquake prone areas under different consolidation ratios

Aldehyde Dehydrogenase-2 Attenuates Myocardial Remodeling and Contractile Dysfunction Induced by a High-Fat Diet

Background/Aims: Consumption of a high-fat (HF) diet exacerbates metabolic cardiomyopathy through lipotoxic mechanisms. In this study, we explored the role of aldehyde dehydrogenase-2 (ALDH2) in myocardial damage induced by a HF diet. Methods: Wild-type C57 BL/6J mice were fed a HF diet or control diet for 16 weeks. ALDH2 overexpression was achieved by injecting a lentiviral ALDH2 expression vector into the left ventricle. Results: Consumption of a HF diet induced metabolic syndrome and myocardial remodeling, and these deleterious effects were attenuated by ALDH2 overexpression. In addition, ALDH2 overexpression attenuated the cellular apoptosis and insulin resistance associated with a HF diet. Mechanistically, ALDH2 overexpression inhibited the expression of c-Jun N-terminal kinase (JNK)-1, activated protein 1 (AP-1), insulin receptor substrate 1 (IRS-1), 4- hydroxynonenal, caspase 3, transforming growth factor β1, and collagen I and III, and enhanced Akt phosphorylation. Conclusion: ALDH2 may effectively attenuate myocardial remodeling and contractile defects induced by a HF diet through the regulation of the JNK/AP-1 and IRS-1/Akt signaling pathways. Our study demonstrates that ALDH2 plays an essential role in protecting cardiac function from lipotoxic cardiomyopathy

Local Memory Search Bat Algorithm for Grey Economic Dynamic System

Control system is a pattern for describing microeconomic performance, so it can provide theory basis for policy-making to make economic performance well and continuously by analyzing and solving the model of economic control system. After analyzing the characteristics of Bat Algorithm (BA), the method to adjust each step of BA is proposed. In the method, each bat took advantage of the optimal location that it had found to guide the direction of search. The result of the case study showed that the proposed algorithm was efficient, then the proposed algorithm was used to solve the grey economic dynamic system, and the results further showed that the method was valid for solving economic control problems. DOI: http://dx.doi.org/10.11591/telkomnika.v11i9.314

Selective Knowledge Distillation for Non-Autoregressive Neural Machine Translation

Benefiting from the sequence-level knowledge distillation, the Non-Autoregressive Transformer (NAT) achieves great success in neural machine translation tasks. However, existing knowledge distillation has side effects, such as propagating errors from the teacher to NAT students, which may limit further improvements of NAT models and are rarely discussed in existing research. In this paper, we introduce selective knowledge distillation by introducing an NAT evaluator to select NAT-friendly targets that are of high quality and easy to learn. In addition, we introduce a simple yet effective progressive distillation method to boost NAT performance. Experiment results on multiple WMT language directions and several representative NAT models show that our approach can realize a flexible trade-off between the quality and complexity of training data for NAT models, achieving strong performances. Further analysis shows that distilling only 5% of the raw translations can help an NAT outperform its counterpart trained on raw data by about 2.4 BLEU