Counterfactually Probing Language Identity in Multilingual Models

Techniques in causal analysis of language models illuminate how linguistic information is organized in LLMs. We use one such technique, AlterRep, a method of counterfactual probing, to explore the internal structure of multilingual models (mBERT and XLM-R). We train a linear classifier on a binary language identity task, to classify tokens between Language X and Language Y. Applying a counterfactual probing procedure, we use the classifier weights to project the embeddings into the null space and push the resulting embeddings either in the direction of Language X or Language Y. Then we evaluate on a masked language modeling task. We find that, given a template in Language X, pushing towards Language Y systematically increases the probability of Language Y words, above and beyond a third-party control language. But it does not specifically push the model towards translation-equivalent words in Language Y. Pushing towards Language X (the same direction as the template) has a minimal effect, but somewhat degrades these models. Overall, we take these results as further evidence of the rich structure of massive multilingual language models, which include both a language-specific and language-general component. And we show that counterfactual probing can be fruitfully applied to multilingual models.Comment: 12 pages, 5 figures, MRL Workshop @ EMNLP 202

Tris(ethyl carbazate-κ2 N,O)nickel(II) dinitrate

The asymmetric unit of the title compound, [Ni(C3H8N2O2)3](NO3)2, contains two independent cations, each built up around a fac-NiN3O3 octa­hedron, and four nitrate anions. Numerous cation-to-anion N—H⋯O hydrogen bonds, some of which are bifurcated, help to establish the packing

Synthesis and characterisation of double-layered octahedral coordination polymers built up from divalent metal ions, mixed carboxylate anions, and ethyl carbazate ligands

Supplementary data related to this article can be found at https://doi.org/10.1016/j.molstruc.2018.12.076.Peer reviewedPostprin

Combined Effects of Simulated Microgravity and Radiation Exposure on Osteoclast Cell Fusion

The loss of bone mass and alteration in bone physiology during space flight are one of the major health risks for astronauts. Although the lack of weight bearing in microgravity is considered a risk factor for bone loss and possible osteoporosis, organisms living in space are also exposed to cosmic radiation and other environmental stress factors. As such, it is still unclear as to whether and by how much radiation exposure contributes to bone loss during space travel, and whether the effects of microgravity and radiation exposure are additive or synergistic. Bone is continuously renewed through the resorption of old bone by osteoclast cells and the formation of new bone by osteoblast cells. In this study, we investigated the combined effects of microgravity and radiation by evaluating the maturation of a hematopoietic cell line to mature osteoclasts. RAW 264.7 monocyte/macrophage cells were cultured in rotating wall vessels that simulate microgravity on the ground. Cells under static 1g or simulated microgravity were exposed to rays of varying doses, and then cultured in receptor activator of nuclear factor-B ligand (RANKL) for the formation of osteoclast giant multinucleated cells (GMCs) and for gene expression analysis. Results of the study showed that radiation alone at doses as low as 0.1 Gy may stimulate osteoclast cell fusion as assessed by GMCs and the expression of signature genes such as tartrate resistant acid phosphatase (Trap) and dendritic cell-specific transmembrane protein (Dcstamp). However, osteoclast cell fusion decreased for doses greater than 0.5 Gy. In comparison to radiation exposure, simulated microgravity induced higher levels of cell fusion, and the effects of these two environmental factors appeared additive. Interestingly, the microgravity effect on osteoclast stimulatory transmembrane protein (Ocstamp) and Dcstamp expressions was significantly higher than the radiation effect, suggesting that radiation may not increase the synthesis of adhesion molecules as much as microgravity

Corrected entropy of BTZ black hole in tunneling approach

We investigate further the recent analysis \cite{R.Banerjee2}, based on a Hamilton-Jacobi type approach, to compute the temperature and entropy of black holes beyond the semiclassical approximation. It is shown how non spherically symmetric geometries are inducted in the general formalism by explicitly considering the BTZ black hole. The leading (logarithmic) and non leading corrections to the area law are obtained.Comment: 12 pages, no figures, minor changes, version to appear in Phys. Lett.

Hardware implementation of intelligent systems

interior, interior vie

Gene-SGAN: a method for discovering disease subtypes with imaging and genetic signatures via multi-view weakly-supervised deep clustering

Disease heterogeneity has been a critical challenge for precision diagnosis and treatment, especially in neurologic and neuropsychiatric diseases. Many diseases can display multiple distinct brain phenotypes across individuals, potentially reflecting disease subtypes that can be captured using MRI and machine learning methods. However, biological interpretability and treatment relevance are limited if the derived subtypes are not associated with genetic drivers or susceptibility factors. Herein, we describe Gene-SGAN - a multi-view, weakly-supervised deep clustering method - which dissects disease heterogeneity by jointly considering phenotypic and genetic data, thereby conferring genetic correlations to the disease subtypes and associated endophenotypic signatures. We first validate the generalizability, interpretability, and robustness of Gene-SGAN in semi-synthetic experiments. We then demonstrate its application to real multi-site datasets from 28,858 individuals, deriving subtypes of Alzheimer's disease and brain endophenotypes associated with hypertension, from MRI and SNP data. Derived brain phenotypes displayed significant differences in neuroanatomical patterns, genetic determinants, biological and clinical biomarkers, indicating potentially distinct underlying neuropathologic processes, genetic drivers, and susceptibility factors. Overall, Gene-SGAN is broadly applicable to disease subtyping and endophenotype discovery, and is herein tested on disease-related, genetically-driven neuroimaging phenotypes