Discovering Low-rank Subspaces for Language-agnostic Multilingual Representations

Large pretrained multilingual language models (ML-LMs) have shown remarkable capabilities of zero-shot cross-lingual transfer, without direct cross-lingual supervision. While these results are promising, follow-up works found that, within the multilingual embedding spaces, there exists strong language identity information which hinders the expression of linguistic factors shared across languages. For semantic tasks like cross-lingual sentence retrieval, it is desired to remove such language identity signals to fully leverage semantic information. In this work, we provide a novel view of projecting away language-specific factors from a multilingual embedding space. Specifically, we discover that there exists a low-rank subspace that primarily encodes information irrelevant to semantics (e.g., syntactic information). To identify this subspace, we present a simple but effective unsupervised method based on singular value decomposition with multiple monolingual corpora as input. Once the subspace is found, we can directly project the original embeddings into the null space to boost language agnosticism without finetuning. We systematically evaluate our method on various tasks including the challenging language-agnostic QA retrieval task. Empirical results show that applying our method consistently leads to improvements over commonly used ML-LMs.Comment: 17 pages, 7 figures, EMNLP 2022 (main conference

Combustion characteristics of lignite char in a fluidized bed under O2/N2, O2/CO2 and O2/H2O atmospheres

As a possible new focus of oxy-fuel work, O2/H2O combustion has many advantages over O2/CO2 combustion, and has gradually gained increasing attention. The unique physicochemical properties (thermal capacity, diffusivity, reactivity) of H2O significantly influence the char combustion characteristics. In the present work, the combustion and kinetics characteristics of lignite char particle were studied in a fluidized bed (FB) reactor under N2, CO2 and H2O atmospheres with different O2 concentrations (15%–27%) and bed temperatures (Tb, 837–937 °C). Results indicated that the average reaction rate (raverage) and the peak reaction rate (rpeak) of lignite char in H2O atmospheres were slower than those in CO2 atmospheres at low O2 concentrations. However, as the O2 concentration increases, the rpeak and raverage of lignite char in H2O atmospheres significantly improved and exceeded those under CO2 atmospheres. The calculation result for the activation energy based on the shrinking-core model showed that the order of activation energy under different atmospheres is: O2/CO2 (28.96 kJ/mol) > O2/H2O (26.11 kJ/mol) > O2/N2 (23.31 kJ/mol). Furthermore, gasification reactions play an important role in both O2/CO2 and O2/H2O combustion, and should not be ignored. As the Tb increased, the active sites occupied by gasification agent were significantly increased, while the active sites occupied by oxygen decreased correspondingly