Relevance Detection and Argumentation Mining in Medical Domain

ABSTRACT In this paper we describe a method to determine the relevancy of a query with a sentence in the document in the field of medical domain. We also describe a method to determine if the given statement supports the query, opposes the query or is neutral with respect to the query. This is a part of CHIS shared task at FIRE 2016

Long Short-Term Memory of Language Models for Predicting Brain Activation During Listening to Stories

International audienceSeveral popular sequence-based and pretrained language models have been found to be successful for text-driven prediction of brain activations. However, these models still lack longterm memory plausibility (i.e. how they deal with long-term dependencies and contextual information) as well as insights on the underlying neural substrate mechanisms. This paper studies the influence of context representations of different language models such as sequence-based models: Long Short-Term Memory networks (LSTMs) and ELMo, and a pretrained Transformer language model (Longformer). In particular, we study how the internal hidden representations align with the brain activity observed via fMRI when the subjects listen to several narrative stories. We use brain imaging recordings of subjects listening to narrative stories to interpret word and sequence embeddings. We further investigate how the representations of language models layers reveal better semantic context during listening. Experiments across all language model representations provide the following cognitive insights: (i) the representations of LSTM cell states are better aligned with brain recordings than LSTM hidden states, the cell state activity can represent more long-term information, (ii) the representations of ELMo and Longformer display a good predictive performance across brain regions for listening stimuli; (iii) Posterior Medial Cortex (PMC), Temporo-Parieto-Occipital junction (TPOJ), and Dorsal Frontal Lobe (DFL) have higher correlation versus Early Auditory (EAC) and Auditory Association Cortex (AAC)

Investigating Long-Term Context of Language Models on Brain Activity during Narrative Listening in fMRI

International audienceAn interesting way to evaluate the representations obtained with machine learning language models is to compared them with human brain recordings. Encoding models have been used to partially predict fMRI recordings of different areas given the features of language models (such as Transformers). However, these models still lack long-term cognitive plausibility as well as insights on the underlying neural substrate mechanisms: e.g. how their representations differ across model layer depth and longer contexts. We study the influence of context representations of different language models such as sequence-based models: Long short-term memory networks (LSTMs), ELMo, and a popular pretrained Transformer language model (Longformer). In particular, we study how the internal hidden representations of such models are aligned with the fMRI brain activity. We use fMRI recordings of subjects listening to narrative stories to interpret word and sequence embedding representations. We further investigate how the representations of language model layers reveal better semantic context during listening. One of the novelties is that we look at several hidden states of LSTMs: cell and output gate states. Our computational experiments provide the following cognitive insights: (i) LSTM cell states are better aligned with brain recordings than LSTM output gate states: the cell state activity can represent more long- term information; (ii) the representations of ELMo and Longformer display a good predictive performance across brain regions for listening stimuli; (iii) Posterior Medial Cortex (PMC), Temporo-Parieto-Occipital junction (TPOJ) and Dorsal Frontal Lobe (DFL) have higher correlation versus Early Auditory (EAC) and Auditory Association Cortex (AAC)

Learning to Parse Sentences with Cross-Situational Learning using Different Word Embeddings Towards Robot Grounding

https://splu-robonlp2021.github.io/International audienceHow pre-trained transformer-based language models perform grounded language acquisition through cross-situational learning (CSL) remains unclear. In particular, it is still not understood how meaning concepts are captured from complex sentences, along with learning language-based interactions, could benefit the field of human-robot interactions and help understand how children learn and ground language. In our current work, we study cross-situational learning to understand the mechanisms enabling children to learn rapidly word-to-meaning mapping with two sequence-based models: (i) Echo State Networks (i.e., Reservoir Computing), and (ii) Long-Short Term Memory Networks (LSTM). We consider the three different input representations: (i) One-Hot encoding, (ii) BERT fine-tuned on Juven+GOLD corpus, and (iii) Google BERT. We investigate which of these three representations better predict the stimulated vision as a function of sentences describing the scenes using two models. Using our approach, we test two datasets: Juven and GoLD, and present how these models generalize only after a few hundred partially described scenes via cross-situational learning. We find that both One-Hot encoding and BERT fine-tuned representations (for both models) significantly improve the predictions. Moreover, we argue that these models are able to learn complex relations between the contexts in which a word appears and their corresponding meaning concepts, handling polysemous and synonymous words. This aspect could be incorporated into a human-robot interaction study that examines grounding language to objects in a physical world and poses a challenge for researchers to investigate better the use of transformer models in robotics and HCI

How distinct are Syntactic and Semantic Representations in the Brain During Sentence Comprehension?

International audienceSyntactic parsing is the task of assigning a syntactic structure to a sentence. Recent works have used syntactic embeddings from constituency trees and other word syntactic features to understand how syntax structure is represented in the brain’s language network. However, the effectiveness of dependency parse trees or the relative predictive power of the three syntax parsers is yet unexplored. We explore syntactic structure embeddings obtained from three parsers and use them in an encoding model to predict brain responses. We use a GCN model (SynGCN embeddings) for the dependency parser that accurately encodes the global syntactic information. Constituency trees explain additional variance better than other syntactic parsing methods. This work was done on data related to English stories only. As we do other kinds of models of language processing in various languages, we want to make similar studies for multi-lingual language

GAE-ISumm: Unsupervised Graph-Based Summarization of Indian Languages

Document summarization aims to create a precise and coherent summary of a text document. Many deep learning summarization models are developed mainly for English, often requiring a large training corpus and efficient pre-trained language models and tools. However, English summarization models for low-resource Indian languages are often limited by rich morphological variation, syntax, and semantic differences. In this paper, we propose GAE-ISumm, an unsupervised Indic summarization model that extracts summaries from text documents. In particular, our proposed model, GAE-ISumm uses Graph Autoencoder (GAE) to learn text representations and a document summary jointly. We also provide a manually-annotated Telugu summarization dataset TELSUM, to experiment with our model GAE-ISumm. Further, we experiment with the most publicly available Indian language summarization datasets to investigate the effectiveness of GAE-ISumm on other Indian languages. Our experiments of GAE-ISumm in seven languages make the following observations: (i) it is competitive or better than state-of-the-art results on all datasets, (ii) it reports benchmark results on TELSUM, and (iii) the inclusion of positional and cluster information in the proposed model improved the performance of summaries.Comment: 9 pages, 7 figure

Visio-Linguistic Brain Encoding

Enabling effective brain-computer interfaces requires understanding how the human brain encodes stimuli across modalities such as visual, language (or text), etc. Brain encoding aims at constructing fMRI brain activity given a stimulus. There exists a plethora of neural encoding models which study brain encoding for single mode stimuli: visual (pretrained CNNs) or text (pretrained language models). Few recent papers have also obtained separate visual and text representation models and performed late-fusion using simple heuristics. However, previous work has failed to explore: (a) the effectiveness of image Transformer models for encoding visual stimuli, and (b) co-attentive multi-modal modeling for visual and text reasoning. In this paper, we systematically explore the efficacy of image Transformers (ViT, DEiT, and BEiT) and multi-modal Transformers (VisualBERT, LXMERT, and CLIP) for brain encoding. Extensive experiments on two popular datasets, BOLD5000 and Pereira, provide the following insights. (1) To the best of our knowledge, we are the first to investigate the effectiveness of image and multi-modal Transformers for brain encoding. (2) We find that VisualBERT, a multi-modal Transformer, significantly outperforms previously proposed single-mode CNNs, image Transformers as well as other previously proposed multi-modal models, thereby establishing new state-of-the-art. The supremacy of visio-linguistic models raises the question of whether the responses elicited in the visual regions are affected implicitly by linguistic processing even when passively viewing images. Future fMRI tasks can verify this computational insight in an appropriate experimental setting.Comment: 18 pages, 13 figure