132 research outputs found
Predefined Sparseness in Recurrent Sequence Models
Inducing sparseness while training neural networks has been shown to yield
models with a lower memory footprint but similar effectiveness to dense models.
However, sparseness is typically induced starting from a dense model, and thus
this advantage does not hold during training. We propose techniques to enforce
sparseness upfront in recurrent sequence models for NLP applications, to also
benefit training. First, in language modeling, we show how to increase hidden
state sizes in recurrent layers without increasing the number of parameters,
leading to more expressive models. Second, for sequence labeling, we show that
word embeddings with predefined sparseness lead to similar performance as dense
embeddings, at a fraction of the number of trainable parameters.Comment: the SIGNLL Conference on Computational Natural Language Learning
(CoNLL, 2018
A Simple Geometric Method for Cross-Lingual Linguistic Transformations with Pre-trained Autoencoders
Powerful sentence encoders trained for multiple languages are on the rise.
These systems are capable of embedding a wide range of linguistic properties
into vector representations. While explicit probing tasks can be used to verify
the presence of specific linguistic properties, it is unclear whether the
vector representations can be manipulated to indirectly steer such properties.
We investigate the use of a geometric mapping in embedding space to transform
linguistic properties, without any tuning of the pre-trained sentence encoder
or decoder. We validate our approach on three linguistic properties using a
pre-trained multilingual autoencoder and analyze the results in both
monolingual and cross-lingual settings
The normalized freebase distance
In this paper, we propose the Normalized Freebase Distance (NFD), a new measure for determing semantic concept relatedness that is based on similar principles as the Normalized Web Distance (NWD). We illustrate that the NFD is more effective when comparing ambiguous concepts
Explaining Character-Aware Neural Networks for Word-Level Prediction: Do They Discover Linguistic Rules?
Character-level features are currently used in different neural network-based
natural language processing algorithms. However, little is known about the
character-level patterns those models learn. Moreover, models are often
compared only quantitatively while a qualitative analysis is missing. In this
paper, we investigate which character-level patterns neural networks learn and
if those patterns coincide with manually-defined word segmentations and
annotations. To that end, we extend the contextual decomposition technique
(Murdoch et al. 2018) to convolutional neural networks which allows us to
compare convolutional neural networks and bidirectional long short-term memory
networks. We evaluate and compare these models for the task of morphological
tagging on three morphologically different languages and show that these models
implicitly discover understandable linguistic rules. Our implementation can be
found at https://github.com/FredericGodin/ContextualDecomposition-NLP .Comment: Accepted at EMNLP 201
Zero-Shot Cross-Lingual Sentiment Classification under Distribution Shift: an Exploratory Study
The brittleness of finetuned language model performance on
out-of-distribution (OOD) test samples in unseen domains has been well-studied
for English, yet is unexplored for multi-lingual models. Therefore, we study
generalization to OOD test data specifically in zero-shot cross-lingual
transfer settings, analyzing performance impacts of both language and domain
shifts between train and test data. We further assess the effectiveness of
counterfactually augmented data (CAD) in improving OOD generalization for the
cross-lingual setting, since CAD has been shown to benefit in a monolingual
English setting. Finally, we propose two new approaches for OOD generalization
that avoid the costly annotation process associated with CAD, by exploiting the
power of recent large language models (LLMs). We experiment with 3 multilingual
models, LaBSE, mBERT, and XLM-R trained on English IMDb movie reviews, and
evaluate on OOD test sets in 13 languages: Amazon product reviews, Tweets, and
Restaurant reviews. Results echo the OOD performance decline observed in the
monolingual English setting. Further, (i) counterfactuals from the original
high-resource language do improve OOD generalization in the low-resource
language, and (ii) our newly proposed cost-effective approaches reach similar
or up to +3.1% better accuracy than CAD for Amazon and Restaurant reviews.Comment: The 3rd Workshop on Multilingual Representation Learning
(MRL@EMNLP2023
The Role of Natural Killer Cells in Sepsis
Severe sepsis and septic shock are still deadly conditions urging to develop novel therapies. A better understanding of the complex modifications of the immune system of septic patients is needed for the development of innovative immunointerventions. Natural killer (NK) cells are characterized as CD3−NKp46+CD56+ cells that can be cytotoxic and/or produce high amounts of cytokines such as IFN-γ. NK cells are also engaged in crosstalks with other immune cells, such as dendritic cells, macrophages, and neutrophils. During the early stage of septic shock, NK cells may play a key role in the promotion of the systemic inflammation, as suggested in mice models. Alternatively, at a later stage, NK cells-acquired dysfunction could favor nosocomial infections and mortality. Standardized biological tools defining patients' NK cell status during the different stages of sepsis are mandatory to guide potential immuno-interventions. Herein, we review the potential role of NK cells during severe sepsis and septic shock
Mechanisms of AXL overexpression and function in Imatinib-resistant chronic myeloid leukemia cells
AXL is a receptor tyrosine kinase of the TAM family, the function of which is poorly understood. We previously identified AXL overexpression in Imatinib (IM)-resistant CML cell lines and patients. The present study was conducted to investigate the role of AXL and the mechanisms underlying AXL overexpression in Tyrosine Kinase Inhibitor (TKI)-resistant CML cells. We present evidence that high AXL expression level is a feature of TKI-resistant CML cells and knockdown of AXL sensitized TKI-resistant cells to IM. In addition, expression of wild-type AXL but not a dominant negative form of AXL confers IM-sensitive CML cells the capacity to resist IM effect. AXL overexpression required PKCα and β and constitutive activation of ERK1/2. Accordingly, GF109203X a PKC inhibitor, U0126 a MEK1 inhibitor and PKCα/β knockdown restore sensitivity to IM while PKCα or PKCβ overexpression in CML cells promotes protection against IM-induced cell death. Finally, using luciferase promoter activity assays we established that AXL is regulated transcriptionally through the AP1 transcription factor. Our findings reveal an unexpected role of AXL in resistance to TKI in CML cells, identify the molecular mechanisms involved in its overexpression and support the notion that AXL is a new marker of resistance to TKI in CML
Daptomycin > 6 mg/kg/day as salvage therapy in patients with complex bone and joint infection: cohort study in a regional reference center
Background: Even if daptomycin does not have approval for the treatment of bone and joint infections (BJI), the Infectious Diseases Society of America guidelines propose this antibiotic as alternative therapy for prosthetic joint infection. The recommended dose is 6 mg/kg/d, whereas recent data support the use of higher doses in these patients.Methods: We performed a cohort study including consecutive patients that have received daptomycin >6 mg/kg/d for complex BJI between 2011 and 2013 in a French regional reference center. Factors associated with treatment failure were determined on univariate Cox analysis and Kaplan-Meier curves.Results: Forty-three patients (age, 61 ± 17 years) received a mean dose of 8 ± 0.9 mg/kg/d daptomycin, for a mean 81 ± 59 days (range, 6-303 days). Most had chronic (n = 37, 86 %) implant-associated (n = 37, 86 %) BJI caused by coagulasenegative staphylococci (n = 32, 74 %). A severe adverse event (SAE) occurred in 6 patients (14 %), including 2 cases of eosinophilic pneumonia, concomitant with daptomycin Cmin >24 mg/L. Outcome was favorable in 30 (77 %) of the 39 clinically assessable patients. Predictors for treatment failure were age, non-optimal surgery and daptomycin withdrawal for SAE.Conclusions: Prolonged high-dose daptomycin therapy was effective in patients with complex BJI. However, optimal surgery remains the cornerstone of medico-surgical strategy; and a higher incidence of eosinophilic pneumonia than expected was recorded
- …