Modeling affirmative and negated action processing in the brain with lexical and compositional semantic models

Recent work shows that distributional semantic models can be used to decode patterns of brain activity associated with individual words and sentence meanings. However, it is yet unclear to what extent such models can be used to study and ecode fMRI patterns associated with specific aspects of semantic composition such as the negation function. In this paper, we apply lexical and compositional semantic models to decode fMRI patterns associated with negated and affirmative sentences containing hand-action verbs. Our results show reduced decoding (correlation) of sentences where the verb is in the negated context, as compared to the affirmative one, within brain regions implicated in action-semantic processing. This supports behavioral and brain imaging studies, suggesting that negation involves reduced access to aspects of the affirmative mental representation. The results pave the way for testing alternate semantic models of negation against human semantic processing in the brain

Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

Multi-head self-attention is a key component of the Transformer, a state-of-the-art architecture for neural machine translation. In this work we evaluate the contribution made by individual attention heads in the encoder to the overall performance of the model and analyze the roles played by them. We find that the most important and confident heads play consistent and often linguistically-interpretable roles. When pruning heads using a method based on stochastic gates and a differentiable relaxation of the L0 penalty, we observe that specialized heads are last to be pruned. Our novel pruning method removes the vast majority of heads without seriously affecting performance. For example, on the English-Russian WMT dataset, pruning 38 out of 48 encoder heads results in a drop of only 0.15 BLEU.Comment: ACL 2019 (camera-ready

Semi-supervised SRL system with Bayesian inference

International audienceWe propose a new approach to perform semi-supervised training of Semantic Role Labeling models with very few amount of initial labeled data. The proposed approach combines in a novel way supervised and unsupervised training, by forcing the supervised classifier to over-generate potential semantic candidates, and then letting unsupervised inference choose the best ones. Hence, the supervised classifier can be trained on a very small corpus and with coarse-grain features, because its precision does not need to be high: its role is mainly to constrain Bayesian inference to explore only a limited part of the full search space. This approach is evaluated on French and English. In both cases, it achieves very good performance and outperforms a strong supervised baseline when only a small number of annotated sentences is available and even without using any previously trained syntactic parser

BICEPP: an example-based statistical text mining method for predicting the binary characteristics of drugs

<p>Abstract</p> <p>Background</p> <p>The identification of drug characteristics is a clinically important task, but it requires much expert knowledge and consumes substantial resources. We have developed a statistical text-mining approach (BInary Characteristics Extractor and biomedical Properties Predictor: BICEPP) to help experts screen drugs that may have important clinical characteristics of interest.</p> <p>Results</p> <p>BICEPP first retrieves MEDLINE abstracts containing drug names, then selects tokens that best predict the list of drugs which represents the characteristic of interest. Machine learning is then used to classify drugs using a document frequency-based measure. Evaluation experiments were performed to validate BICEPP's performance on 484 characteristics of 857 drugs, identified from the Australian Medicines Handbook (AMH) and the PharmacoKinetic Interaction Screening (PKIS) database. Stratified cross-validations revealed that BICEPP was able to classify drugs into all 20 major therapeutic classes (100%) and 157 (of 197) minor drug classes (80%) with areas under the receiver operating characteristic curve (AUC) > 0.80. Similarly, AUC > 0.80 could be obtained in the classification of 173 (of 238) adverse events (73%), up to 12 (of 15) groups of clinically significant cytochrome P450 enzyme (CYP) inducers or inhibitors (80%), and up to 11 (of 14) groups of narrow therapeutic index drugs (79%). Interestingly, it was observed that the keywords used to describe a drug characteristic were not necessarily the most predictive ones for the classification task.</p> <p>Conclusions</p> <p>BICEPP has sufficient classification power to automatically distinguish a wide range of clinical properties of drugs. This may be used in pharmacovigilance applications to assist with rapid screening of large drug databases to identify important characteristics for further evaluation.</p

Adaptive Radiation within Marine Anisakid Nematodes: A Zoogeographical Modeling of Cosmopolitan, Zoonotic Parasites

Parasites of the nematode genus Anisakis are associated with aquatic organisms. They can be found in a variety of marine hosts including whales, crustaceans, fish and cephalopods and are known to be the cause of the zoonotic disease anisakiasis, a painful inflammation of the gastro-intestinal tract caused by the accidental consumptions of infectious larvae raw or semi-raw fishery products. Since the demand on fish as dietary protein source and the export rates of seafood products in general is rapidly increasing worldwide, the knowledge about the distribution of potential foodborne human pathogens in seafood is of major significance for human health. Studies have provided evidence that a few Anisakis species can cause clinical symptoms in humans. The aim of our study was to interpolate the species range for every described Anisakis species on the basis of the existing occurrence data. We used sequence data of 373 Anisakis larvae from 30 different hosts worldwide and previously published molecular data (n = 584) from 53 field-specific publications to model the species range of Anisakis spp., using a interpolation method that combines aspects of the alpha hull interpolation algorithm as well as the conditional interpolation approach. The results of our approach strongly indicate the existence of species-specific distribution patterns of Anisakis spp. within different climate zones and oceans that are in principle congruent with those of their respective final hosts. Our results support preceding studies that propose anisakid nematodes as useful biological indicators for their final host distribution and abundance as they closely follow the trophic relationships among their successive hosts. The modeling might although be helpful for predicting the likelihood of infection in order to reduce the risk of anisakiasis cases in a given area