A Quantitative Approach to Understand Self-Supervised Models as Cross-lingual Feature Extractors

In this work, we study the features extracted by English self-supervised learning (SSL) models in cross-lingual contexts and propose a new metric to predict the quality of feature representations. Using automatic speech recognition (ASR) as a downstream task, we analyze the effect of model size, training objectives, and model architecture on the models' performance as a feature extractor for a set of topologically diverse corpora. We develop a novel metric, the Phonetic-Syntax Ratio (PSR), to measure the phonetic and synthetic information in the extracted representations using deep generalized canonical correlation analysis. Results show the contrastive loss in the wav2vec2.0 objective facilitates more effective cross-lingual feature extraction. There is a positive correlation between PSR scores and ASR performance, suggesting that phonetic information extracted by monolingual SSL models can be used for downstream tasks in cross-lingual settings. The proposed metric is an effective indicator of the quality of the representations and can be useful for model selection.Comment: 12 pages, 5 figures, 4 table

Condensing Multilingual Knowledge with Lightweight Language-Specific Modules

Incorporating language-specific (LS) modules is a proven method to boost performance in multilingual machine translation. This approach bears similarity to Mixture-of-Experts (MoE) because it does not inflate FLOPs. However, the scalability of this approach to hundreds of languages (experts) tends to be unmanageable due to the prohibitive number of parameters introduced by full-rank matrices in fully-connected layers. In this work, we introduce the Language-Specific Matrix Synthesis (LMS) method. This approach constructs LS modules by generating low-rank matrices from two significantly smaller matrices to approximate the full-rank matrix. Furthermore, we condense multilingual knowledge from multiple LS modules into a single shared module with the Fuse Distillation (FD) technique to improve the efficiency of inference and model serialization. We show that our LMS method significantly outperforms previous LS methods and MoE methods with the same amount of extra parameters, e.g., 1.73 BLEU points over the Switch Transformer on many-to-many multilingual machine translation. Importantly, LMS is able to have comparable translation performance with much fewer parameters.Comment: Accepted at the main conference of EMNLP 202

Opportunistic Intermittent Control with Safety Guarantees for Autonomous Systems

Control schemes for autonomous systems are often designed in a way that anticipates the worst case in any situation. At runtime, however, there could exist opportunities to leverage the characteristics of specific environment and operation context for more efficient control. In this work, we develop an online intermittent-control framework that combines formal verification with model-based optimization and deep reinforcement learning to opportunistically skip certain control computation and actuation to save actuation energy and computational resources without compromising system safety. Experiments on an adaptive cruise control system demonstrate that our approach can achieve significant energy and computation savings

Dermatophagoides farinae microRNAs released to external environments via exosomes regulate inflammation-related gene expression in human bronchial epithelial cells

BackgroundDermatophagoides farinae (DFA) is an important species of house dust mites (HDMs) that causes allergic diseases. Previous studies have focused on allergens with protein components to explain the allergic effect of HDMs; however, there is little knowledge on the role of microRNAs (miRNAs) in the allergic effect of HDMs. This study aimed to unravel the new mechanism of dust mite sensitization from the perspective of cross-species transport of extracellular vesicles-encapsulated miRNAs from HDMs.MethodsSmall RNA (sRNA) sequencing was performed to detect miRNAs expression profiles from DFA, DFA-derived exosomes and DFA culture supernatants. A quantitative fluorescent real-time PCR (qPCR) assay was used to detect miRNAs expression in dust specimens. BEAS-2B cells endocytosed exosomes were modeled in vitro to detect miRNAs from DFA and the expression of related inflammatory factors. Representative dfa-miR-276-3p and dfa-novel-miR2 were transfected into BEAS-2B cells, and then differentially expressed genes (DEGs) were analyzed by RNA sequencing. Protein-protein interaction (PPI) network analysis and Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway and Gene Ontology (GO) terms enrichment analyses were performed on the first 300 nodes of DEGs.ResultssRNA sequencing identified 42 conserved miRNAs and 66 novel miRNAs in DFA, DFA-derived exosomes, and DFA culture supernatants. A homology analysis was performed on the top 18 conserved miRNAs with high expression levels. The presence of dust mites and miRNAs from HDMs in living environment were also validated. Following uptake of DFA-derived exosomes by BEAS-2B cells, exosomes transported miRNAs from DFA to target cells and produced pro-inflammatory effects in corresponding cells. RNA sequencing identified DEGs in dfa-miR-276-3p and dfa-novel-miR2 transfected BEAS-2B cells. GO and KEGG enrichment analyses revealed the role of exosomes with cross-species transporting of DFA miRNAs in inflammatory signaling pathways, such as JAK-STAT signaling pathway, PI3K/AKT signaling pathway and IL-6-mediated signaling pathway.ConclusionOur findings demonstrate the miRNAs expression profiles in DFA for the first time. The DFA miRNAs are delivered into living environments via exosomes, and engulfed by human bronchial epithelial cells, and cross-species regulation may contribute to inflammation-related processes

A hierarchical study for urban statistical indicators on the prevalence of COVID-19 in Chinese city clusters based on multiple linear regression (MLR) and polynomial best subset regression (PBSR) analysis

With evidence-based measures, COVID-19 can be effectively controlled by advanced data analysis and prediction. However, while valuable insights are available, there is a shortage of robust and rigorous research on what factors shape COVID-19 transmissions at the city cluster level. Therefore, to bridge the research gap, we adopted a data-driven hierarchical modeling approach to identify the most influential factors in shaping COVID-19 transmissions across different Chinese cities and clusters. The data used in this study are from Chinese officials, and hierarchical modeling conclusions drawn from the analysis are systematic, multifaceted, and comprehensive. To further improve research rigor, the study utilizes SPSS, Python and RStudio to conduct multiple linear regression and polynomial best subset regression (PBSR) analysis for the hierarchical modeling. The regression model utilizes the magnitude of various relative factors in nine Chinese city clusters, including 45 cities at a different level of clusters, to examine these aspects from the city cluster scale, exploring the correlation between various factors of the cities. These initial 12 factors are comprised of ‘Urban population ratio’, ‘Retail sales of consumer goods’, ‘Number of tourists’, ‘Tourism Income’, ‘Ratio of the elderly population (> 60 year old) in this city’, ‘population density’, ‘Mobility scale (move in/inbound) during the spring festival’, ‘Ratio of Population and Health facilities’, ‘Jobless rate (%)’, ‘The straight-line distance from original epicenter Wuhan to this city’, ‘urban per capita GDP’, and ‘the prevalence of the COVID-19’. The study’s results provide rigorously-tested and evidence-based insights on most instrumental factors that shape COVID-19 transmissions across cities and regions in China. Overall, the study findings found that per capita GDP and population mobility rates were the most affected factors in the prevalence of COVID-19 in a city, which could inform health experts and government officials to design and develop evidence-based and effective public health policies that could curb the spread of the COVID-19 pandemic

A tumor microenvironment-responsive micelle co-delivered radiosensitizer Dbait and doxorubicin for the collaborative chemo-radiotherapy of glioblastoma

Glioblastoma is rather recalcitrant to existing therapies and effective interventions are needed. Here we report a novel microenvironment-responsive micellar system (ch-K5(s-s)R8-An) for the co-delivery of the radiosensitizer Dbait and the chemotherapeutic doxorubicin (DOX) to glioblastoma. Accordingly, the ch-K5(s-s)R8-An/(Dbait-DOX) micelles plus radiotherapy (RT) treatment resulted in a high degree of apoptosis and DNA damage, which significantly reduced cell viability and proliferation capacity of U251 cells to 64.0% and 16.3%, respectively. The angiopep-2-modified micelles exhibited substantial accumulation in brain-localized U251 glioblastoma xenografts in mice compared to angiopep-2-lacking micelles. The ch-K5(s-s)R8-An/(Dbait-DOX) + RT treatment group exhibited the smallest tumor size and most profound tumor tissue injury in orthotopic U251 tumors, leading to an increase in median survival time of U251 tumor-bearing mice from 26 days to 56 days. The ch-K5(s-s)R8-An/(Dbait-DOX) micelles can be targeted to brain-localized U251 tumor xenografts and sensitize the tumor to chemotherapy and radiotherapy, thereby overcoming the inherent therapeutic challenges associated with malignant glioblastoma

Characterization of Lenticulostriate Arteries and Its Associations With Vascular Risk Factors in Community-Dwelling Elderly

Lenticulostriate arteries (LSAs) supply blood to important subcortical areas and are, therefore, essential for maintaining the optimal functioning of the brain’s most metabolically active nuclei. Past studies have demonstrated the potential for quantifying the morphology of LSAs as biomarkers of vascular fragility or underlying arteriopathies. Thus, the current study aims to evaluate the morphological features of LSAs, their potential value in cerebrovascular risk stratification, and their concordance with other vascular risk factors in community-dwelling elderly people. A total of 125 community-dwelling elderly subjects who underwent a brain MRI scan were selected from our prospectively collected imaging database. The morphological measures of LSAs were calculated on the vascular skeletons obtained by manual tracing, and the number of LSAs was counted. Additionally, imaging biomarkers of small vessel disease were evaluated, and the diameters of major cerebral arteries were measured. The effects of vascular risk factors on LSA morphometry, as well as the relationship between LSA measures and other imaging biomarkers, were investigated. We found that smokers had shorter (p = 0.04) and straighter LSAs (p < 0.01) compared to nonsmokers, and the presence of hypertension is associated with less tortuous LSAs (p = 0.03) in community-dwelling elderly. Moreover, the middle cerebral artery diameter was positively correlated with LSA count (r = 0.278, p = 0.025) and vessel tortuosity (r = 0.257, p = 0.04). The posterior cerebral artery diameter was positively correlated with vessel tortuosity and vessel length. Considering the scarcity of noninvasive methods for measuring small artery abnormalities in the brain, the LSA morphological measures may provide valuable information to better understand cerebral small vessel degeneration during aging