CEDR: Contextualized Embeddings for Document Ranking

Although considerable attention has been given to neural ranking architectures recently, far less attention has been paid to the term representations that are used as input to these models. In this work, we investigate how two pretrained contextualized language models (ELMo and BERT) can be utilized for ad-hoc document ranking. Through experiments on TREC benchmarks, we find that several existing neural ranking architectures can benefit from the additional context provided by contextualized language models. Furthermore, we propose a joint approach that incorporates BERT's classification vector into existing neural models and show that it outperforms state-of-the-art ad-hoc ranking baselines. We call this joint approach CEDR (Contextualized Embeddings for Document Ranking). We also address practical challenges in using these models for ranking, including the maximum input length imposed by BERT and runtime performance impacts of contextualized language models.Comment: Appeared in SIGIR 2019, 4 page

New international commercial courts:a delocalized approach

New international commercial courts can be analysed by examining how their features differ from those of their domestic counterpart courts and from those of international commercial arbitration. This conceptual tool is termed delocalization. Higher and lower levels of featural differences, or delocalization, may affect a new court’s reception, whether local actors can participate in the new court and the new court’s relations with the domestic courts. These factors influence the extent and speed of a new court’s integration into the legal landscape as an institutional transplant. A delocalization analysis can also help track the new and domestic courts’ continuing influence over each other and the adoption, sharing or abandonment of features over time

Adapting Learned Sparse Retrieval for Long Documents

Learned sparse retrieval (LSR) is a family of neural retrieval methods that transform queries and documents into sparse weight vectors aligned with a vocabulary. While LSR approaches like Splade work well for short passages, it is unclear how well they handle longer documents. We investigate existing aggregation approaches for adapting LSR to longer documents and find that proximal scoring is crucial for LSR to handle long documents. To leverage this property, we proposed two adaptations of the Sequential Dependence Model (SDM) to LSR: ExactSDM and SoftSDM. ExactSDM assumes only exact query term dependence, while SoftSDM uses potential functions that model the dependence of query terms and their expansion terms (i.e., terms identified using a transformer's masked language modeling head). Experiments on the MSMARCO Document and TREC Robust04 datasets demonstrate that both ExactSDM and SoftSDM outperform existing LSR aggregation approaches for different document length constraints. Surprisingly, SoftSDM does not provide any performance benefits over ExactSDM. This suggests that soft proximity matching is not necessary for modeling term dependence in LSR. Overall, this study provides insights into handling long documents with LSR, proposing adaptations that improve its performance.Comment: SIGIR 202

A Unified Framework for Learned Sparse Retrieval

Learned sparse retrieval (LSR) is a family of first-stage retrieval methods that are trained to generate sparse lexical representations of queries and documents for use with an inverted index. Many LSR methods have been recently introduced, with Splade models achieving state-of-the-art performance on MSMarco. Despite similarities in their model architectures, many LSR methods show substantial differences in effectiveness and efficiency. Differences in the experimental setups and configurations used make it difficult to compare the methods and derive insights. In this work, we analyze existing LSR methods and identify key components to establish an LSR framework that unifies all LSR methods under the same perspective. We then reproduce all prominent methods using a common codebase and re-train them in the same environment, which allows us to quantify how components of the framework affect effectiveness and efficiency. We find that (1) including document term weighting is most important for a method's effectiveness, (2) including query weighting has a small positive impact, and (3) document expansion and query expansion have a cancellation effect. As a result, we show how removing query expansion from a state-of-the-art model can reduce latency significantly while maintaining effectiveness on MSMarco and TripClick benchmarks. Our code is publicly available at https://github.com/thongnt99/learned-sparse-retrieva

RSDD-Time: Temporal Annotation of Self-Reported Mental Health Diagnoses

Self-reported diagnosis statements have been widely employed in studying language related to mental health in social media. However, existing research has largely ignored the temporality of mental health diagnoses. In this work, we introduce RSDD-Time: a new dataset of 598 manually annotated self-reported depression diagnosis posts from Reddit that include temporal information about the diagnosis. Annotations include whether a mental health condition is present and how recently the diagnosis happened. Furthermore, we include exact temporal spans that relate to the date of diagnosis. This information is valuable for various computational methods to examine mental health through social media because one's mental health state is not static. We also test several baseline classification and extraction approaches, which suggest that extracting temporal information from self-reported diagnosis statements is challenging.Comment: 6 pages, accepted for publication at the CLPsych workshop at NAACL-HLT 201

Characterizing Question Facets for Complex Answer Retrieval

Complex answer retrieval (CAR) is the process of retrieving answers to questions that have multifaceted or nuanced answers. In this work, we present two novel approaches for CAR based on the observation that question facets can vary in utility: from structural (facets that can apply to many similar topics, such as 'History') to topical (facets that are specific to the question's topic, such as the 'Westward expansion' of the United States). We first explore a way to incorporate facet utility into ranking models during query term score combination. We then explore a general approach to reform the structure of ranking models to aid in learning of facet utility in the query-document term matching phase. When we use our techniques with a leading neural ranker on the TREC CAR dataset, our methods rank first in the 2017 TREC CAR benchmark, and yield up to 26% higher performance than the next best method.Comment: 4 pages; SIGIR 2018 Short Pape

Barendskraal, a diverse amniote locality from the Lystrosaurus Assemblage Zone, Early Triassic of South Africa

Main articleA diverse amniote fauna has been recovered from Lower Triassic Lystrosaurus Assemblage Zone exposures on the farm Barendskraal, near Middelburg in Eastern Cape Province, South Africa. The fauna includes the dicynodont therapsid Lystrosaurus sp., the therocephalian therapsids Tetracynodon darti, Moschorhinus kitchingi and Ericiolacerta parva, the archosauromorph reptiles Proterosuchus fergusi and Prolacerta broomi, and the procolophonoid reptiles Owenetta kitchingorum, Sauropareion anoplus and Saurodectes rogersorum. The locality is remarkable in that although it is fossil-rich, Lystrosaurus fossils do not appear to be as abundant as elsewhere in this assemblage zone, and the diversity of taxa at Barendskraal (at least nine species) is surpassed only by that of the famous HarrismithCommonage locality in the northeastern Free State province (at least 13 species). However, the fauna at Harrismith Commonage is typical of most other Lystrosaurus biozone localities in being dominated numerically by Lystrosaurus. Study of the tetrapod taxa from Barendskraal is providing new insights into procolophonoid phylogeny and survivorship across the Permo-Triassic boundary, as well as the stratigraphic ranges of various taxa in the Lower Triassic deposits of the Karoo Basin.National Geographic Society grant number 6929-00

The Suppression of Immune System Disorders by Passive Attrition

Exposure to infectious diseases has an unexpected benefit of inhibiting autoimmune diseases and allergies. This is one of many fundamental fitness tradeoffs associated with immune system architecture. The immune system attacks pathogens, but also may (inappropriately) attack the host. Exposure to pathogens can suppress the deleterious response, at the price of illness and the decay of immunity to previous diseases. This “hygiene hypothesis” has been associated with several possible underlying biological mechanisms. This study focuses on physiological constraints that lead to competition for survival between immune system cell types. Competition maintains a relatively constant total number of cells within each niche. The constraint implies that adding cells conferring new immunity requires loss (passive attrition) of some cells conferring previous immunities. We consider passive attrition as a mechanism to prevent the initial proliferation of autoreactive cells, thus preventing autoimmune disease. We see that this protection is a general property of homeostatic regulation and we look specifically at both the IL-15 and IL-7 regulated niches to make quantitative predictions using a mathematical model. This mathematical model yields insight into the dynamics of the “Hygiene Hypothesis,” and makes quantitative predictions for experiments testing the ability of passive attrition to suppress immune system disorders. The model also makes a prediction of an anti-correlation between prevalence of immune system disorders and passive attrition rates

SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health Conditions

Mental health is a significant and growing public health concern. As language usage can be leveraged to obtain crucial insights into mental health conditions, there is a need for large-scale, labeled, mental health-related datasets of users who have been diagnosed with one or more of such conditions. In this paper, we investigate the creation of high-precision patterns to identify self-reported diagnoses of nine different mental health conditions, and obtain high-quality labeled data without the need for manual labelling. We introduce the SMHD (Self-reported Mental Health Diagnoses) dataset and make it available. SMHD is a novel large dataset of social media posts from users with one or multiple mental health conditions along with matched control users. We examine distinctions in users' language, as measured by linguistic and psychological variables. We further explore text classification methods to identify individuals with mental conditions through their language.Comment: COLING 201

Real supermodels wear wool: summarizing the impact of the pregnant sheep as an animal model for adaptive fetal programming

• Intrauterine growth restriction (IUGR) continues to be a global epidemic that is associated with high early-life mortality rates and greater risk for developing metabolic disorders that lower length and quality of life in affected individuals. • Fetal programming of muscle growth and metabolic function associated with IUGR is often comparable among nonlitter bearing mammalian species, which allows much of the information learned in domestic animal models to be applicable to humans (and other animals). • Recent studies in sheep models of IUGR have begun to uncover the molecular mechanisms linking adaptive fetal programming and metabolic dysfunction. • Targets of adaptive fetal programming indicated by sheep studies include adrenergic and inflammatory pathways that regulate skeletal muscle growth and glucose metabolism. Adaptive changes in these pathways represent potential focus areas for prenatal interventions or postnatal treatments to improve outcomes in IUGR-born offspring