Cascaded Multi-task Adaptive Learning Based on Neural Architecture Search

Cascading multiple pre-trained models is an effective way to compose an end-to-end system. However, fine-tuning the full cascaded model is parameter and memory inefficient and our observations reveal that only applying adapter modules on cascaded model can not achieve considerable performance as fine-tuning. We propose an automatic and effective adaptive learning method to optimize end-to-end cascaded multi-task models based on Neural Architecture Search (NAS) framework. The candidate adaptive operations on each specific module consist of frozen, inserting an adapter and fine-tuning. We further add a penalty item on the loss to limit the learned structure which takes the amount of trainable parameters into account. The penalty item successfully restrict the searched architecture and the proposed approach is able to search similar tuning scheme with hand-craft, compressing the optimizing parameters to 8.7% corresponding to full fine-tuning on SLURP with an even better performance

Plugin Speech Enhancement: A Universal Speech Enhancement Framework Inspired by Dynamic Neural Network

The expectation to deploy a universal neural network for speech enhancement, with the aim of improving noise robustness across diverse speech processing tasks, faces challenges due to the existing lack of awareness within static speech enhancement frameworks regarding the expected speech in downstream modules. These limitations impede the effectiveness of static speech enhancement approaches in achieving optimal performance for a range of speech processing tasks, thereby challenging the notion of universal applicability. The fundamental issue in achieving universal speech enhancement lies in effectively informing the speech enhancement module about the features of downstream modules. In this study, we present a novel weighting prediction approach, which explicitly learns the task relationships from downstream training information to address the core challenge of universal speech enhancement. We found the role of deciding whether to employ data augmentation techniques as crucial downstream training information. This decision significantly impacts the expected speech and the performance of the speech enhancement module. Moreover, we introduce a novel speech enhancement network, the Plugin Speech Enhancement (Plugin-SE). The Plugin-SE is a dynamic neural network that includes the speech enhancement module, gate module, and weight prediction module. Experimental results demonstrate that the proposed Plugin-SE approach is competitive or superior to other joint training methods across various downstream tasks

An improved adaptive genetic algorithm for image segmentation and vision alignment used in microelectronic bonding

In order to improve the precision and efficiency of microelectronic bonding, this paper presents an improved adaptive genetic algorithm (IAGA) for the image segmentation and vision alignment of the solder joints in the microelectronic chips. The maximum between-cluster variance (OTSU) threshold segmentation method was adopted for the image segmentation of microchips, and the IAGA was introduced to the threshold segmentation considering the features of the images. The performance of the image segmentation was investigated by computational and experimental tests. The results show that the IAGA has faster convergence and better global optimality compared with standard genetic algorithm (SGA), and the quality of the segmented images becomes better by using the OTSU threshold segmentation method based on IAGA. On the basis of moment invariant approach, the microvision alignment was realized. Experiments were carried out to implement the microvision alignment of the solder joints in the microelectronic chips, and the results indicate that there are no alignment failures using the OTSU threshold segmentation method based on IAGA, which is superior to the OTSU method based on SGA in improving the precision and speed of the vision alignments

VE-KWS: Visual Modality Enhanced End-to-End Keyword Spotting

The performance of the keyword spotting (KWS) system based on audio modality, commonly measured in false alarms and false rejects, degrades significantly under the far field and noisy conditions. Therefore, audio-visual keyword spotting, which leverages complementary relationships over multiple modalities, has recently gained much attention. However, current studies mainly focus on combining the exclusively learned representations of different modalities, instead of exploring the modal relationships during each respective modeling. In this paper, we propose a novel visual modality enhanced end-to-end KWS framework (VE-KWS), which fuses audio and visual modalities from two aspects. The first one is utilizing the speaker location information obtained from the lip region in videos to assist the training of multi-channel audio beamformer. By involving the beamformer as an audio enhancement module, the acoustic distortions, caused by the far field or noisy environments, could be significantly suppressed. The other one is conducting cross-attention between different modalities to capture the inter-modal relationships and help the representation learning of each modality. Experiments on the MSIP challenge corpus show that our proposed model achieves 2.79% false rejection rate and 2.95% false alarm rate on the Eval set, resulting in a new SOTA performance compared with the top-ranking systems in the ICASSP2022 MISP challenge.Comment: 5 pages. Accepted at ICASSP202

Learning to Check Contract Inconsistencies

Contract consistency is important in ensuring the legal validity of the contract. In many scenarios, a contract is written by filling the blanks in a precompiled form. Due to carelessness, two blanks that should be filled with the same (or different)content may be incorrectly filled with different (or same) content. This will result in the issue of contract inconsistencies, which may severely impair the legal validity of the contract. Traditional methods to address this issue mainly rely on manual contract review, which is labor-intensive and costly. In this work, we formulate a novel Contract Inconsistency Checking (CIC) problem, and design an end-to-end framework, called Pair-wise Blank Resolution (PBR), to solve the CIC problem with high accuracy. Our PBR model contains a novel BlankCoder to address the challenge of modeling meaningless blanks. BlankCoder adopts a two-stage attention mechanism that adequately associates a meaningless blank with its relevant descriptions while avoiding the incorporation of irrelevant context words. Experiments conducted on real-world datasets show the promising performance of our method with a balanced accuracy of 94.05% and an F1 score of 90.90% in the CIC problem.Comment: Accepted by AAAI 202

E2F1 Suppresses Oxidative Metabolism and Endothelial Differentiation of Bone Marrow Progenitor Cells

RATIONALE: The majority of current cardiovascular cell therapy trials use bone marrow progenitor cells (BM PCs) and achieve only modest efficacy; the limited potential of these cells to differentiate into endothelial-lineage cells is one of the major barriers to the success of this promising therapy. We have previously reported that the E2F transcription factor 1 (E2F1) is a repressor of revascularization after ischemic injury. OBJECTIVE: We sought to define the role of E2F1 in the regulation of BM PC function. METHODS AND RESULTS: Ablation of E2F1 (E2F1 deficient) in mouse BM PCs increases oxidative metabolism and reduces lactate production, resulting in enhanced endothelial differentiation. The metabolic switch in E2F1-deficient BM PCs is mediated by a reduction in the expression of pyruvate dehydrogenase kinase 4 and pyruvate dehydrogenase kinase 2; overexpression of pyruvate dehydrogenase kinase 4 reverses the enhancement of oxidative metabolism and endothelial differentiation. Deletion of E2F1 in the BM increases the amount of PC-derived endothelial cells in the ischemic myocardium, enhances vascular growth, reduces infarct size, and improves cardiac function after myocardial infarction. CONCLUSION: Our results suggest a novel mechanism by which E2F1 mediates the metabolic control of BM PC differentiation, and strategies that inhibit E2F1 or enhance oxidative metabolism in BM PCs may improve the effectiveness of cell therapy