CIF-PT: Bridging Speech and Text Representations for Spoken Language Understanding via Continuous Integrate-and-Fire Pre-Training

Speech or text representation generated by pre-trained models contains modal-specific information that could be combined for benefiting spoken language understanding (SLU) tasks. In this work, we propose a novel pre-training paradigm termed Continuous Integrate-and-Fire Pre-Training (CIF-PT). It relies on a simple but effective frame-to-token alignment: continuous integrate-and-fire (CIF) to bridge the representations between speech and text. It jointly performs speech-to-text training and language model distillation through CIF as the pre-training (PT). Evaluated on SLU benchmark SLURP dataset, CIF-PT outperforms the state-of-the-art model by 1.94% of accuracy and 2.71% of SLU-F1 on the tasks of intent classification and slot filling, respectively. We also observe the cross-modal representation extracted by CIF-PT obtains better performance than other neural interfaces for the tasks of SLU, including the dominant speech representation learned from self-supervised pre-training.Comment: Accepted by ACL 2023 Finding

Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System

Large-scale Language Models (LLMs) are constrained by their inability to process lengthy inputs. To address this limitation, we propose the Self-Controlled Memory (SCM) system to unleash infinite-length input capacity for large-scale language models. Our SCM system is composed of three key modules: the language model agent, the memory stream, and the memory controller. The language model agent iteratively processes ultra-long inputs and stores all historical information in the memory stream. The memory controller provides the agent with both long-term memory (archived memory) and short-term memory (flash memory) to generate precise and coherent responses. The controller determines which memories from archived memory should be activated and how to incorporate them into the model input. Our SCM system can be integrated with any LLMs to enable them to process ultra-long texts without any modification or fine-tuning. Experimental results show that our SCM system enables LLMs, which are not optimized for multi-turn dialogue, to achieve multi-turn dialogue capabilities that are comparable to ChatGPT, and to outperform ChatGPT in scenarios involving ultra-long document summarization or long-term conversations. Additionally, we will supply a test set, which covers common long-text input scenarios, for evaluating the abilities of LLMs in processing long documents.~\footnote{Working in progress.}\footnote{\url{https://github.com/wbbeyourself/SCM4LLMs}}Comment: Working in progres

High-efficient deep learning-based DTI reconstruction with flexible diffusion gradient encoding scheme

Purpose: To develop and evaluate a novel dynamic-convolution-based method called FlexDTI for high-efficient diffusion tensor reconstruction with flexible diffusion encoding gradient schemes. Methods: FlexDTI was developed to achieve high-quality DTI parametric mapping with flexible number and directions of diffusion encoding gradients. The proposed method used dynamic convolution kernels to embed diffusion gradient direction information into feature maps of the corresponding diffusion signal. Besides, our method realized the generalization of a flexible number of diffusion gradient directions by setting the maximum number of input channels of the network. The network was trained and tested using data sets from the Human Connectome Project and a local hospital. Results from FlexDTI and other advanced tensor parameter estimation methods were compared. Results: Compared to other methods, FlexDTI successfully achieves high-quality diffusion tensor-derived variables even if the number and directions of diffusion encoding gradients are variable. It increases peak signal-to-noise ratio (PSNR) by about 10 dB on Fractional Anisotropy (FA) and Mean Diffusivity (MD), compared with the state-of-the-art deep learning method with flexible diffusion encoding gradient schemes. Conclusion: FlexDTI can well learn diffusion gradient direction information to achieve generalized DTI reconstruction with flexible diffusion gradient schemes. Both flexibility and reconstruction quality can be taken into account in this network.Comment: 11 pages,6 figures,3 table

Fuzzy Multi-Objectives Topology Optimization of Slider Pallet in the Picking Machine of Camellia Fruit

In order to improve the dynamic characteristics of the slider pallet in the camellia fruit picking machine under the traditional empirical design and to lighten the weight, a fuzzy multi-objective topology optimization design method was proposed. In this paper, a static and dynamic topology optimization mathematical model was constructed by the compromise programming method, and the weight coefficients of each sub-objective were dynamically assigned by the fuzzy satisfaction variable weight coefficient method, and then the fuzzy multi-objective topology optimization design of the slider pallet for bending condition, bending-torsional complex condition, inertia condition and the first three orders of dynamic frequency was performed. The optimization results showed that the weight of the optimized slider pallet was reduced by 19.4%, and the first-order modal frequency was increased by 5.0%, second order modal frequency increased by 6.6%, third order modal frequency increased by 8.2%; the maximum deformation and maximum stress were increased, but still met the design requirements

Severe Acute Respiratory Syndrome, Beijing, 2003

The largest outbreak of severe acute respiratory syndrome (SARS) struck Beijing in spring 2003. Multiple importations of SARS to Beijing initiated transmission in several healthcare facilities. Beijing’s outbreak began March 5; by late April, daily hospital admissions for SARS exceeded 100 for several days; 2,521 cases of probable SARS occurred. Attack rates were highest in those 20–39 years of age; 1% of cases occurred in children <10 years. The case-fatality rate was highest among patients >65 years (27.7% vs. 4.8% for those 20–64 years, p < 0.001). Healthcare workers accounted for 16% of probable cases. The proportion of case-patients without known contact to a SARS patient increased significantly in May. Implementation of early detection, isolation, contact tracing, quarantine, triage of case-patients to designated SARS hospitals, and community mobilization ended the outbreak

Simultaneous optical and radar observations of poleward moving auroral forms under different IMF conditions

Using high temporal resolution optical data obtained from three-wavelength all-sky imagers at Chinese Yellow River Station in the Arctic, together with the EISCAT Svalbard radar (ESR) and SuperDARN radars, we investigated the dayside poleward moving auroral forms (PMAFs) and the associated plasma features in the polar ionosphere under different interplanetary magnetic field (IMF) conditions, between 0900 and 1010 UT on 22 December 2003. Simultaneous optical and ESR observations revealed that all PMAFs were clearly associated with pulsed particle precipitations. During northward IMF, particles can precipitate into lower altitudes and reach the ionospheric E-region, and there is a reverse convection cell associated with these PMAFs. This cell is one of the typical signatures of the dayside high-latitude (lobe) reconnection in the polar ionosphere. These results indicate that the PMAFs were associated with the high-latitude reconnection. During southward IMF, the PMAFs show larger latitudinal motion, indicating a longer mean lifetime, and the associated ionospheric features indicate that the PMAFs were generated by the dayside low-latitude reconnection

Predicting potential microbe-disease associations with graph attention autoencoder, positive-unlabeled learning, and deep neural network

BackgroundMicrobes have dense linkages with human diseases. Balanced microorganisms protect human body against physiological disorders while unbalanced ones may cause diseases. Thus, identification of potential associations between microbes and diseases can contribute to the diagnosis and therapy of various complex diseases. Biological experiments for microbe–disease association (MDA) prediction are expensive, time-consuming, and labor-intensive.MethodsWe developed a computational MDA prediction method called GPUDMDA by combining graph attention autoencoder, positive-unlabeled learning, and deep neural network. First, GPUDMDA computes disease similarity and microbe similarity matrices by integrating their functional similarity and Gaussian association profile kernel similarity, respectively. Next, it learns the feature representation of each microbe–disease pair using graph attention autoencoder based on the obtained disease similarity and microbe similarity matrices. Third, it selects a few reliable negative MDAs based on positive-unlabeled learning. Finally, it takes the learned MDA features and the selected negative MDAs as inputs and designed a deep neural network to predict potential MDAs.ResultsGPUDMDA was compared with four state-of-the-art MDA identification models (i.e., MNNMDA, GATMDA, LRLSHMDA, and NTSHMDA) on the HMDAD and Disbiome databases under five-fold cross validations on microbes, diseases, and microbe-disease pairs. Under the three five-fold cross validations, GPUDMDA computed the best AUCs of 0.7121, 0.9454, and 0.9501 on the HMDAD database and 0.8372, 0.8908, and 0.8948 on the Disbiome database, respectively, outperforming the other four MDA prediction methods. Asthma is the most common chronic respiratory condition and affects ~339 million people worldwide. Inflammatory bowel disease is a class of globally chronic intestinal disease widely existed in the gut and gastrointestinal tract and extraintestinal organs of patients. Particularly, inflammatory bowel disease severely affects the growth and development of children. We used the proposed GPUDMDA method and found that Enterobacter hormaechei had potential associations with both asthma and inflammatory bowel disease and need further biological experimental validation.ConclusionThe proposed GPUDMDA demonstrated the powerful MDA prediction ability. We anticipate that GPUDMDA helps screen the therapeutic clues for microbe-related diseases