Global Normalization of Convolutional Neural Networks for Joint Entity and Relation Classification

We introduce globally normalized convolutional neural networks for joint entity classification and relation extraction. In particular, we propose a way to utilize a linear-chain conditional random field output layer for predicting entity types and relations between entities at the same time. Our experiments show that global normalization outperforms a locally normalized softmax layer on a benchmark dataset.Comment: EMNLP 201

Exploring Different Dimensions of Attention for Uncertainty Detection

Neural networks with attention have proven effective for many natural language processing tasks. In this paper, we develop attention mechanisms for uncertainty detection. In particular, we generalize standardly used attention mechanisms by introducing external attention and sequence-preserving attention. These novel architectures differ from standard approaches in that they use external resources to compute attention weights and preserve sequence information. We compare them to other configurations along different dimensions of attention. Our novel architectures set the new state of the art on a Wikipedia benchmark dataset and perform similar to the state-of-the-art model on a biomedical benchmark which uses a large set of linguistic features.Comment: accepted at EACL 201

GradSim: Gradient-Based Language Grouping for Effective Multilingual Training

Most languages of the world pose low-resource challenges to natural language processing models. With multilingual training, knowledge can be shared among languages. However, not all languages positively influence each other and it is an open research question how to select the most suitable set of languages for multilingual training and avoid negative interference among languages whose characteristics or data distributions are not compatible. In this paper, we propose GradSim, a language grouping method based on gradient similarity. Our experiments on three diverse multilingual benchmark datasets show that it leads to the largest performance gains compared to other similarity measures and it is better correlated with cross-lingual model performance. As a result, we set the new state of the art on AfriSenti, a benchmark dataset for sentiment analysis on low-resource African languages. In our extensive analysis, we further reveal that besides linguistic features, the topics of the datasets play an important role for language grouping and that lower layers of transformer models encode language-specific features while higher layers capture task-specific information

NLNDE at SemEval-2023 Task 12: Adaptive Pretraining and Source Language Selection for Low-Resource Multilingual Sentiment Analysis

This paper describes our system developed for the SemEval-2023 Task 12 "Sentiment Analysis for Low-resource African Languages using Twitter Dataset". Sentiment analysis is one of the most widely studied applications in natural language processing. However, most prior work still focuses on a small number of high-resource languages. Building reliable sentiment analysis systems for low-resource languages remains challenging, due to the limited training data in this task. In this work, we propose to leverage language-adaptive and task-adaptive pretraining on African texts and study transfer learning with source language selection on top of an African language-centric pretrained language model. Our key findings are: (1) Adapting the pretrained model to the target language and task using a small yet relevant corpus improves performance remarkably by more than 10 F1 score points. (2) Selecting source languages with positive transfer gains during training can avoid harmful interference from dissimilar languages, leading to better results in multilingual and cross-lingual settings. In the shared task, our system wins 8 out of 15 tracks and, in particular, performs best in the multilingual evaluation

Intramuscular DNA Vaccination of Juvenile Carp against Spring Viremia of Carp Virus Induces Full Protection and Establishes a Virus-Specific B and T Cell Response

Although spring viremia of carp virus (SVCV) can cause high mortalities in common carp, a commercial vaccine is not available for worldwide use. Here, we report a DNA vaccine based on the expression of the SVCV glycoprotein (G) which, when injected in the muscle even at a single low dose of 0.1 µg DNA/g of fish, confers up to 100% protection against a subsequent bath challenge with SVCV. Importantly, to best validate vaccine efficacy, we also optimized a reliable bath challenge model closely mimicking a natural infection, based on a prolonged exposure of carp to SVCV at 15°C. Using this optimized bath challenge, we showed a strong age-dependent susceptibility of carp to SVCV, with high susceptibility at young age (3 months) and a full resistance at 9 months. We visualized local expression of the G protein and associated early inflammatory response by immunohistochemistry and described changes in the gene expression of pro-inflammatory cytokines, chemokines, and antiviral genes in the muscle of vaccinated fish. Adaptive immune responses were investigated by analyzing neutralizing titers against SVCV in the serum of vaccinated fish and the in vitro proliferation capacity of peripheral SVCV-specific T cells. We show significantly higher serum neutralizing titers and the presence of SVCV-specific T cells in the blood of vaccinated fish, which proliferated upon stimulation with SVCV. Altogether, this is the first study reporting on a protective DNA vaccine against SVCV in carp and the first to provide a detailed characterization of local innate as well as systemic adaptive immune responses elicited upon DNA vaccination that suggest a role not only of B cells but also of T cells in the protection conferred by the SVCV-G DNA vaccine

Phylogeny of the Viral Hemorrhagic Septicemia Virus in European Aquaculture

<p>One of the most valuable aquaculture fish in Europe is the rainbow trout, Oncorhynchus mykiss, but the profitability of trout production is threatened by a highly lethal infectious disease, viral hemorrhagic septicemia (VHS), caused by the VHS virus (VHSV). For the past few decades, the subgenogroup Ia of VHSV has been the main cause of VHS outbreaks in European freshwater-farmed rainbow trout. Little is currently known, however, about the phylogenetic radiation of this Ia lineage into subordinate Ia clades and their subsequent geographical spread routes. We investigated this topic using the largest Ia-isolate dataset ever compiled, comprising 651 complete G gene sequences: 209 GenBank Ia isolates and 442 Ia isolates from this study. The sequences come from 11 European countries and cover the period 1971-2015. Based on this dataset, we documented the extensive spread of the Ia population and the strong mixing of Ia isolates, assumed to be the result of the Europe-wide trout trade. For example, the Ia lineage underwent a radiation into nine Ia clades, most of which are difficult to allocate to a specific geographic distribution. Furthermore, we found indications for two rapid, large-scale population growth events, and identified three polytomies among the Ia clades, both of which possibly indicate a rapid radiation. However, only about 4% of Ia haplotypes (out of 398) occur in more than one European country. This apparently conflicting finding regarding the Europe-wide spread and mixing of Ia isolates can be explained by the high mutation rate of VHSV. Accordingly, the mean period of occurrence of a single Ia haplotype was less than a full year, and we found a substitution rate of up to 7.813 × 10^-4 nucleotides per site per year. Finally, we documented significant differences between Germany and Denmark regarding their VHS epidemiology, apparently due to those countries' individual handling of VHS.</p