Predicting Multi-Codebook Vector Quantization Indexes for Knowledge Distillation

Knowledge distillation(KD) is a common approach to improve model performance in automatic speech recognition (ASR), where a student model is trained to imitate the output behaviour of a teacher model. However, traditional KD methods suffer from teacher label storage issue, especially when the training corpora are large. Although on-the-fly teacher label generation tackles this issue, the training speed is significantly slower as the teacher model has to be evaluated every batch. In this paper, we reformulate the generation of teacher label as a codec problem. We propose a novel Multi-codebook Vector Quantization (MVQ) approach that compresses teacher embeddings to codebook indexes (CI). Based on this, a KD training framework (MVQ-KD) is proposed where a student model predicts the CI generated from the embeddings of a self-supervised pre-trained teacher model. Experiments on the LibriSpeech clean-100 hour show that MVQ-KD framework achieves comparable performance as traditional KD methods (l1, l2), while requiring 256 times less storage. When the full LibriSpeech dataset is used, MVQ-KD framework results in 13.8% and 8.2% relative word error rate reductions (WERRs) for non -streaming transducer on test-clean and test-other and 4.0% and 4.9% for streaming transducer. The implementation of this work is already released as a part of the open-source project icefall.Comment: Submitted to ICASSP 202

Spatio-temporal divergence in the responses of Finland's boreal forests to climate variables

Spring greening in boreal forest ecosystems has been widely linked to increasing temperature, but few studies have attempted to unravel the relative effects of climate variables such as maximum temperature (TMX), minimum temperature (TMN), mean temperature (TMP), precipitation (PRE) and radiation (RAD) on vegetation growth at different stages of growing season. However, clarifying these effects is fundamental to better understand the relationship between vegetation and climate change. This study investigated spatio-temporal divergence in the responses of Finland's boreal forests to climate variables using the plant phenology index (PPI) calculated based on the latest Collection V006 MODIS BRDF-corrected surface reflectance products (MCD43C4) from 2002 to 2018, and identified the dominant climate variables controlling vegetation change during the growing season (May-September) on a monthly basis. Partial least squares (PLS) regression was used to quantify the response of PPI to climate variables and distinguish the separate impacts of different variables. The study results show the dominant effects of temperature on the PPI in May and June, with TMX, TMN and TMP being the most important explanatory variables for the variation of PPI depending on the location, respectively. Meanwhile, drought had an unexpectedly positive impact on vegetation in few areas. More than 50 % of the variation of PPI could be explained by climate variables for 68.5 % of the entire forest area in May and 87.7 % in June, respectively. During July to September, the PPI variance explained by climate and corresponding spatial extent rapidly decreased. Nevertheless, the RAD was found be the most important explanatory variable to July PPI in some areas. In contrast, the PPI in August and September was insensitive to climate in almost all of the regions studied. Our study gives useful insights on quantifying and identifying the relative importance of climate variables to boreal forest, which can be used to predict the possible response of forest under future warming.Peer reviewe

A Review on Ultrafast-Laser Power Bed Fusion Technology

Additive manufacturing of metals by employing continuous wave and short pulse lasers completely changes the way of modern industrial production. But the ultrafast laser has the superiority to short pulse laser and continuous wave laser in additive manufacturing. It has higher peak power, small thermal effect, high machining accuracy and low damage threshold. It can effectively perform additive manufacturing for special materials and improve the mechanical properties of parts. This article reviews the mechanism of the interaction between ultrafast laser and metal materials to rule the manufacturing processes. The current application of ultrafast laser on forming and manufacturing special materials, including refractory metals, transparent materials, composite materials and high thermal conductivity materials are also discussed. Among the review, the shortcomings and challenges of the current experimental methods are discussed as well. Finally, suggestions are provided for the industrial application of ultrashort pulse laser in the field of additive manufacturing in the future

A Review on Ultrafast-Laser Power Bed Fusion Technology

Additive manufacturing of metals by employing continuous wave and short pulse lasers completely changes the way of modern industrial production. But the ultrafast laser has the superiority to short pulse laser and continuous wave laser in additive manufacturing. It has higher peak power, small thermal effect, high machining accuracy and low damage threshold. It can effectively perform additive manufacturing for special materials and improve the mechanical properties of parts. This article reviews the mechanism of the interaction between ultrafast laser and metal materials to rule the manufacturing processes. The current application of ultrafast laser on forming and manufacturing special materials, including refractory metals, transparent materials, composite materials and high thermal conductivity materials are also discussed. Among the review, the shortcomings and challenges of the current experimental methods are discussed as well. Finally, suggestions are provided for the industrial application of ultrashort pulse laser in the field of additive manufacturing in the future

Microgrid Optimal Dispatch Based on Distributed Economic Model Predictive Control Algorithm

A microgrid cluster is composed of multiple interconnected microgrids and operates in the form of cluster, which can realize energy complementation between microgrids and significantly improve their renewable energy consumption capacity and system operation reliability. A microgrid optimal dispatch based on a distributed economic model predictive control algorithm is proposed in this paper. Firstly, the control task of the microgrid power generation system is defined, which is required to meet the load demand while reducing the economic loss of the system and realize dynamic economic optimization. The global objective function is designed based on the control task, and the detailed design method of the distributed economic model predictive controller is given. The control law is obtained by an iterative calculation using the Nash optimal method, which can effectively reduce the amount of data in the communication network. Finally, a microgrid group composed of four microgrids is used as an example for simulation verification. The simulation results show that the distributed economic model predictive control algorithm proposed in this paper has good economic benefits for microgrid dispatching

Metal-Organic Framework as a Simple and General Inert Nanocarrier for Photosensitizers to Implement Activatable Photodynamic Therapy

10.1002/adfm.201707519Advanced Functional Materials281

What is the optimal dose of adipose-derived mesenchymal stem cells treatment for knee osteoarthritis? A conventional and network meta-analysis of randomized controlled trials

Abstract Background Despite increasing clinical investigations underscoring the efficacy and safety of adipose-derived mesenchymal stem cells (AD-MSCs) therapy in knee osteoarthritis (KOA), no article has recently reviewed the cell dosage. This study aimed to evaluate the efficacy and safety of varying doses of AD-MSCs in treating KOA using conventional and network meta-analysis. Methods A search of databases in in Chinese and English was performed to identify randomized controlled trials (RCT) on MSCs for knee osteoarthritis from the inception date to May 1, 2022. This study mainly analyzed the efficacy of AD-MSCs in the treatment of KOA, and subgroup analysis was performed on the therapeutic effects of MSCs from different tissues at the same dose. We divided the different cell doses into low, moderate, and high groups, with the corresponding cell doses: (0–25)*10^6, (25–50)*10^6, and > 50*10^6 cells, respectively. We further analyzed the improvement of improvement of the Visual Analog Scale (VAS) and the Western Ontario and McMaster Universities Arthritis Index (WOMAC) scores and the incidence of adverse events (AEs) after varied dosage injection. Results A total of 16 literatures were included in this study, of which 8 literatures were about AD-MSCs. Conventional meta-analysis suggests that AD-MSCs can reduce pain and improve function in KOA patients, regardless of the cell doses, up to 12 months of follow-up. The network meta-analysis showed that intra-articular injection of AD-MSCs significantly improved pain and knee function scores in KOA patients compared with the control group at 3, 6, and 12 months. Among the three groups, the high-dose group had the best treatment effect, and the degree of joint pain and dysfunction indicators improved more significantly in the early stage. For adverse events, there was a dose–response trend that increased with increasing doses. Conclusions Both cell doses reduced pain and improved knee function in KOA patients. The effect surpassed in the high-dose group than in the moderate-dose, low-dose and control groups. However, adverse events also increase with the increase in dose, which should be carefully considered in clinical application, and the side effects still need to be paid attention to. Considering the limitations of this meta-analysis, future studies need to further explore the efficacy and safety of different doses of treatment, and carry out large sample, multi-center, randomized controlled trials to ensure the reliability and promotion value of the research results

Glutamine and Intestinal Physiology and Pathology

International audienc