Predicting Multi-Codebook Vector Quantization Indexes for Knowledge Distillation

Knowledge distillation(KD) is a common approach to improve model performance in automatic speech recognition (ASR), where a student model is trained to imitate the output behaviour of a teacher model. However, traditional KD methods suffer from teacher label storage issue, especially when the training corpora are large. Although on-the-fly teacher label generation tackles this issue, the training speed is significantly slower as the teacher model has to be evaluated every batch. In this paper, we reformulate the generation of teacher label as a codec problem. We propose a novel Multi-codebook Vector Quantization (MVQ) approach that compresses teacher embeddings to codebook indexes (CI). Based on this, a KD training framework (MVQ-KD) is proposed where a student model predicts the CI generated from the embeddings of a self-supervised pre-trained teacher model. Experiments on the LibriSpeech clean-100 hour show that MVQ-KD framework achieves comparable performance as traditional KD methods (l1, l2), while requiring 256 times less storage. When the full LibriSpeech dataset is used, MVQ-KD framework results in 13.8% and 8.2% relative word error rate reductions (WERRs) for non -streaming transducer on test-clean and test-other and 4.0% and 4.9% for streaming transducer. The implementation of this work is already released as a part of the open-source project icefall.Comment: Submitted to ICASSP 202

Microstructure, Inclusions, and Elemental Distribution of a Compacted Graphite Iron Alloyed by Ce and La Rare Earth (RE) Elements

This work investigates the microstructure and inclusions of a compacted graphite iron (CGI) alloyed by Ce and La rare earth (RE) elements. In our study, alloying elemental distribution and solute segregation were characterized by methods of secondary ion mass spectrometry (SIMS) and a three-dimensional atom probe (3DAP) with high sensitivity and spatial resolution. RE sulfide, MgS, carbide, and composite inclusions formed during solidification and provided heterogeneous nucleation cores for the nucleation of the graphite. Significant solute clustering in the matrix, coupled with the segregation of solute to grain boundaries, was observed. C, Mn, Cr, and V were soluted in cementite and promoted the precipitation of cementite, while Si was found to be soluted in ferrite. Cu is usually distributed uniformly in ferrite, but some Cu-rich atom clusters were observed to segregate towards the interface between the ferrite and cementite, stabilizing the pearlite. In addition, P, as a segregation element, was enriched along the boundaries continuously. The RE elements participated in the formation of inclusions, consuming harmful elements such as As and P, and also promoted the heterogeneous nucleation of the graphite and segregated, in the form of solute atoms, at its interfaces

Development of an efficient thermal transfer structure in rapid heat cycle molding

Conference Name:International Conference on Materials Science and Technology. Conference Address: Jeju Island, SOUTH KOREA. Time:DEC 27-28, 2010.The efficiency of the orientation thermal transfer is the key in rapid heat cycle molding (RHCM) technology, because it significantly affects the energy consumption, productivity and the quality of the final polymer parts. Therefore, the thermal response of the integral mold insert in SRHCM process has been simulated by ANSYS, and a novel thermal transfer structure in the form of combined mold insert with insulation layer has been developed to reduce the energy waste. The milled U-grooves act as thermal transfer channels in this structure, which can be manufactured conveniently to obtain high surface quality easily. The simulation results show that the novel structure can save energy consumption evidently, that the heating time is reduced by 35.7 percent, and that the cooling time is reduced by 24.9 percent compared with the unmodified one

Eco-efficiency analysis of industrial system in China: A data envelopment analysis approach

Eco-efficiency is an instrument for sustainability analysis, indicating how efficient the economic activity is with regard to nature's goods and services. This paper conducts an eco-efficiency analysis for regional industrial systems in China by developing data envelopment analysis (DEA) based models. Using real data of 30 provinces in China, an empirical study is employed to illustrate the pattern of regional industrial systems' eco-efficiency. The results indicate that Tianjing, Shanghai, Guangdong, Beijing, Hainan and Qinghai are relatively eco-efficient. The results also show that, provinces with higher level GDP per capita will have higher eco-efficiency relatively with an exception of Hainan and Qinghai. The study provides deeper insights into the causes of eco-inefficiency, and gives further implications on environmental protection strategies in China. In the article, we also discuss the advantages and disadvantages of using DEA in eco-efficiency analysis and areas that require further work are presented.Eco-efficiency Industrial system Data envelopment analysis (DEA) China

Numerical Simulation of Rock Failure Process with a 3D Grain-Based Rock Model

A grain-based rock model was developed and applied to study mechanical characteristics and failure micromechanics in thick-walled cylinder and wellbore stability tests. The rock is represented as an assembly of tetrahedral blocks with bonded contacts. Material heterogeneity is modeled by varying the tensile strength at the block contacts. This grain-based rock model differs from previous disk/sphere particle-based rock models in its ability to represent a zero (or very low) initial porosity condition, as well as highly interlocked irregular block shapes that provide resistance to movement even after contact breakage. As a result, this model can reach higher uniaxial compressive strength to tensile strength ratios and larger friction coefficients than the disk/sphere particle-based rock model. The model captured the rock fragmentation process near the wellbore due to buckling and spalling. Thin fragments of rock similar to onion skins were produced, as observed in laboratory breakout experiments. The results suggest that this approach may be well suited to study the rock disaggregation process and other geomechanical problems in the rock excavation

Topology Optimization on Bracket Side Panel of Beach Cleaner's Dumping Device

Conference Name:International Symposium on Manufacturing Systems Engineering (ISMSE 2011). Conference Address: Hong Kong, PEOPLES R CHINA. Time:SEP 17-18, 2011.According to the typical working conditions of beach cleaner, which come from the result of the virtual prototyping model's simulation analysis. Establish the optimal topology design of the bracket side panel of the dumping device by ANSYS software. Compare the results before and after optimization, we conclude that, the bracket side panel has been not only reduce by 17.1% in mass under meeting the strength and rigidity requirements but also improved its welding technology. At the same time, this paper can provide reference for the lightening design and manufacturability improvement of complex loading parts

Microstructure, Inclusions, and Elemental Distribution of a Compacted Graphite Iron Alloyed by Ce and La Rare Earth (RE) Elements

This work investigates the microstructure and inclusions of a compacted graphite iron (CGI) alloyed by Ce and La rare earth (RE) elements. In our study, alloying elemental distribution and solute segregation were characterized by methods of secondary ion mass spectrometry (SIMS) and a three-dimensional atom probe (3DAP) with high sensitivity and spatial resolution. RE sulfide, MgS, carbide, and composite inclusions formed during solidification and provided heterogeneous nucleation cores for the nucleation of the graphite. Significant solute clustering in the matrix, coupled with the segregation of solute to grain boundaries, was observed. C, Mn, Cr, and V were soluted in cementite and promoted the precipitation of cementite, while Si was found to be soluted in ferrite. Cu is usually distributed uniformly in ferrite, but some Cu-rich atom clusters were observed to segregate towards the interface between the ferrite and cementite, stabilizing the pearlite. In addition, P, as a segregation element, was enriched along the boundaries continuously. The RE elements participated in the formation of inclusions, consuming harmful elements such as As and P, and also promoted the heterogeneous nucleation of the graphite and segregated, in the form of solute atoms, at its interfaces

Study on the Development of an Infectious Disease-Specific Health Literacy Scale in the Chinese Population

To develop a scale to assess infectious disease-specific health literacy (IDSHL) in China and test its initial psychometric properties. Methods: Item pooling, reduction and assessment of psychometric properties were conducted. The scale was divided into 2 subscales; subscale 1 assessed an individual\u27s skills to prevent/treat infectious diseases and subscale 2 assessed cognitive ability. In 2014, 9000 people aged 15-69 years were randomly sampled from 3 provinces and asked to complete the IDSHL questionnaire. Cronbach\u27s á was calculated to assess reliability. Exploratory factor analysis, t-test, correlations, receiver operating characteristic (ROC) curve and logistic regression were used to examine validity. Results: Each of the 22 items in subscale 1 had a content validity index \u3e0.8. In total, 8858 people completed the scale. The principal components factor analysis suggested a 5-factor solution. All factor loadings were \u3e0.40 ( p\u3c0.05). The IDSHL score was 22.07±7.91 (mean±SD; total score=38.62). Significant differences were observed across age (r=.0.276), sex (males: 21.65±8.03; females: 22.47±7.78), education (14.16±8.19 to 26.55±6.26), 2-week morbidity (present: 20.62±8.17, absent: 22.35±7.83; p\u3c0.001) and health literacy of the highest and lowest 27% score groups (all p\u3c0.05). The ROC curve indicated that 76.2% of respondents were adequate in IDSHL. Binary logistic regression analysis revealed 12 predictors of IDSHL adequacy (p\u3c0.05). Among the 22 remaining items, Corrected Item-Total Correlation ranged from 0.316 to 0.504 and Cronbach\u27s á values ranged from 0.754 to 0.810 if the items were deleted. The overall á value was 0.839 and the difficulty coefficient ranged from 1.19 to 4.08. For subscale 2, there were statistically significant differences between the mean scores of those with a correct/incorrect answer (all p\u3c0.001). Conclusions: The newly developed 28-item scale provides an efficient, psychometrically sound and userfriendly measure of IDSHL in the Chinese population