Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains

Pre-trained language models have been applied to various NLP tasks with considerable performance gains. However, the large model sizes, together with the long inference time, limit the deployment of such models in real-time applications. One line of model compression approaches considers knowledge distillation to distill large teacher models into small student models. Most of these studies focus on single-domain only, which ignores the transferable knowledge from other domains. We notice that training a teacher with transferable knowledge digested across domains can achieve better generalization capability to help knowledge distillation. Hence we propose a Meta-Knowledge Distillation (Meta-KD) framework to build a meta-teacher model that captures transferable knowledge across domains and passes such knowledge to students. Specifically, we explicitly force the meta-teacher to capture transferable knowledge at both instance-level and feature-level from multiple domains, and then propose a meta-distillation algorithm to learn single-domain student models with guidance from the meta-teacher. Experiments on public multi-domain NLP tasks show the effectiveness and superiority of the proposed Meta-KD framework. Further, we also demonstrate the capability of Meta-KD in the settings where the training data is scarce

A review on fundamentals for designing hydrogen evolution electrocatalyst

As a clean, efficient, and renewable energy source, hydrogen has always been recognized as a favourable replacement of fossil fuel. A primary challenge is an efficient generation of hydrogen to fulfil the requirements of hydrogen on a commercial scale. The electrocatalytic process of HER (hydrogen evolution reaction), as primary phase in water electrolytic process for H2 production, has undergone comprehensive observation from recent decades. Electrolytic water splitting presents a promised route to attain efficient hydrogen generation concerning energy conversion and storage, with electrolysis or catalysis playing a pivotal role. The advancement of catalyst or electrocatalysts that are effective, enduring and economical is necessary prerequisite for realizing the intended electrolytic hydrogen generation from water splitting for applicable considerations, embodying the primary emphasis of this article. In this extensive review, we initially summarize the basics of the Hydrogen evolution reaction and examine the latest cutting-edge progress in economical and highly efficiency catalysts utilizing both non-noble and noble metals. Moreover, the recent breakthroughs over the preceding years in electrolytic HER employing more affordable and widely available nanoparticles with a specific center of attention on economical and non-platinum electrocatalysts rooted in metal free (MF) and transition metal composite catalysts are deliberated here

Qwen Technical Report

Large language models (LLMs) have revolutionized the field of artificial intelligence, enabling natural language processing tasks that were previously thought to be exclusive to humans. In this work, we introduce Qwen, the first installment of our large language model series. Qwen is a comprehensive language model series that encompasses distinct models with varying parameter counts. It includes Qwen, the base pretrained language models, and Qwen-Chat, the chat models finetuned with human alignment techniques. The base language models consistently demonstrate superior performance across a multitude of downstream tasks, and the chat models, particularly those trained using Reinforcement Learning from Human Feedback (RLHF), are highly competitive. The chat models possess advanced tool-use and planning capabilities for creating agent applications, showcasing impressive performance even when compared to bigger models on complex tasks like utilizing a code interpreter. Furthermore, we have developed coding-specialized models, Code-Qwen and Code-Qwen-Chat, as well as mathematics-focused models, Math-Qwen-Chat, which are built upon base language models. These models demonstrate significantly improved performance in comparison with open-source models, and slightly fall behind the proprietary models.Comment: 59 pages, 5 figure

Comparative risk of cardiac arrhythmias associated with acetylcholinesterase inhibitor use

Background: Acquired long-QT syndrome (ALQTS) and its associated condition, torsades de pointes (TdP – a malignant cardiac arrhythmia), are associated with the use of certain medications. Case reports and pharmacodynamic studies suggest donepezil, one of the three acetylcholinesterase inhibitors (AChEIs) used in the treatment of Alzheimer’s Disease (AD) and related dementias, may be associated with a greater risk of ALQTS and malignant arrhythmias. Only a limited number of studies have generated relevant information, and no population-based epidemiologic studies have directly examined comparative risk between AChEIs. Methods: Using Canadian hospitalization and prescription medication administrative databases – the Discharge Abstract Database (DAD) and National Prescription Drug Utilization Information System (NPDUIS) respectively – I included individuals in seven jurisdictions (British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Prince Edward Island, Newfoundland and Labrador) between April 1st 2011 and January 31st 2019. Included adults were aged 66 and over, and either initiated the use of donepezil, galantamine, oral rivastigmine, or transdermal rivastigmine (defined as no previous dispensation of AChEI recorded in NPDUIS in 365 days prior). The outcome of a hospitalization for malignant arrhythmia was identified by ICD-10-CA codes I47.2 and I49.00. The hospitalization date set as the 15th of the month in primary analysis (with adjustment to first or end of month in sensitivity analyses) and a multivariable Cox regression model was fitted to estimate the hazard of a hospitalization for malignant arrhythmia, dependent upon AChEI use. The primary analysis assessed the time to hospitalization for malignant arrhythmia using the primary DAD diagnostic field and a maximum available follow-up of eight years. In secondary analyses, I included malignant arrhythmia codes from any diagnostic field and limited follow-up to 365 days. Variables adjusted for include demographic covariates and comorbidities identified from previous hospitalizations and prescription medication use. Results: The cohort included 162,527 patients (mean age 82, 40% male; median days follow-up 386). Most subjects (n = 127,038; 78%) were treated with donepezil, while 25,582 (16%) received galantamine and 9,907 (6%) received rivastigmine. During a median follow up of 1.06 years, I identified 90 hospitalizations for a malignant arrhythmia – including 58 in the donepezil group (23 per 100,000 person-years) and 32 for other AChEIs (44 per 100,000 person-years). After adjustment for confounding, initiation of donepezil was associated with a 45% lower hazard of hospitalization (adjusted HR 0.55, 95% CI 0.36 to 0.85) relative to other AChEIs as a group. When assessing for malignant arrhythmias occurring in any diagnosis field, I identified 336 events (258 in the donepezil group and 78 in other AChEIs group); however, donepezil initiation was no longer significantly associated with hospitalizations (adjusted HR 1.01, 95% CI 0.78 to 1.30). My findings were similar in an analysis limited to 365 days of follow-up, and in sensitivity analyses modifying the hospitalization date definition (since only month/year of hospitalization was provided). Conclusion: In this large, population-based cohort study, initiation of donepezil was not associated with an increased risk of hospitalization for malignant arrhythmias in comparison to other AChEIs. This was not consistent with case reports and pharmacodynamic studies. Given that adjustment for confounders moved the HR towards the null, residual confounding, caused by uncaptured comorbidities may have caused the lower risk in the donepezil group (e.g., since donepezil is a first line AChEI for treatment of Alzheimer’s). Further research is warranted

Stabilization and Speed Control of a Permanent Magnet Synchronous Motor with Dual-Rotating Rotors

The permanent magnet synchronous motor (PMSM) with dual-rotating rotors is a typical nonlinear multi-variable coupled system. It is sensitive to load disturbances and the change of interior parameters. The traditional proportional-integral (PI) controller is widely used in the speed control of a motor because of its simplicity; however, it cannot meet the requirements needed for high performance. In addition, when the loads of both of the rotors change, it is difficult to ensure that the system runs stably. With an aim to mitigate these problems, a method called master-slave motor control is proposed to guarantee the stability of the motor system in all cases. And then, a speed controller is designed to eliminate the influence of uncertain terms. The proposed control strategy is implemented both in simulations and in experiments. Through the analysis and comparison of the proportional-integral (PI) controller and the sliding-mode controller, the effectiveness of the proposed control strategy is validated

The impact of natural resource, information and communication technology adoption, and economic expansion on financial development in post COVID era

This study explores the interplay between natural resources, economic growth, and financial development in China, extending the inquiry to the moderating role of Information and Communication Technology (ICT). Utilizing the advanced Fourier Autoregressive Distributed Lag (ARDL) framework on data spanning from 1991 to 2020, our analysis delineates the dual influence of natural resources on financial development. Results indicate that natural resources alone have a deleterious effect on financial development both in the short and long run. However, when combined with ICT, the long-run effects are positive, suggesting that ICT has a transformative potential on the natural resources-financial development nexus. Conversely, in the short run, this interaction proves detrimental. These findings underscore the imperative for a holistic strategy that harmonizes economic growth, ICT advancement, and natural resource utilization to foster sustainable financial development

Fatigue Analysis of Composite Bolted Joints under Random and Constant Amplitude Fatigue Loadings

This paper attempts to analyze the random fatigue life and failure modes of joints using two calculation methods. Three kinds of tests were carried out, which were the static test, constant amplitude fatigue test and the random fatigue test, and four kinds of joints were designed. After the static test, the joint was subjected to a constant amplitude fatigue test by selecting different percentages of load according to the static strength. In order to predict the random fatigue life more precisely, two calculation methods were carried out, which were the linear cumulative damage method and the equivalent loading finite element method. Based on the linear cumulative damage hypothesis, the fatigue life of the joint was established as a function of the load amplitude, and then, the random life prediction was calculated by the amplitude distribution of the random loading. Another method was the equivalent loading method, which was to obtain the equivalent constant amplitude fatigue loading of the random loading spectrum. The finite element model was established based on the stiffness and strength degradation rule. The equivalent random life and fatigue failure modes of the joint were modeled. The two life prediction methods show good agreement with the fatigue experimental result, and all prediction results were included in a scatter band of the factor of 2