Using Deep Neural Networks to Learn Syntactic Agreement

We consider the extent to which different deep neural network (DNN) configurations can learn syntactic relations, by taking up Linzen et al.’s (2016) work on subject-verb agreement with LSTM RNNs. We test their methods on a much larger corpus than they used (a ⇠24 million example part of the WaCky corpus, instead of their ⇠1.35 million example corpus, both drawn from Wikipedia). We experiment with several different DNN architectures (LSTM RNNs, GRUs, and CNNs), and alternative parameter settings for these systems (vocabulary size, training to test ratio, number of layers, memory size, drop out rate, and lexical embedding dimension size). We also try out our own unsupervised DNN language model. Our results are broadly compatible with those that Linzen et al. report. However, we discovered some interesting, and in some cases, surprising features of DNNs and language models in their performance of the agreement learning task. In particular, we found that DNNs require large vocabularies to form substantive lexical embeddings in order to learn structural patterns. This finding has interesting consequences for our understanding of the way in which DNNs represent syntactic information. It suggests that DNNs learn syntactic patterns more efficiently through rich lexical embeddings, with semantic as well as syntactic cues, than from training on lexically impoverished strings that highlight structural patterns

‘Synthetic cannabis’: A dangerous misnomer

The term 'synthetic cannabis' has been widely used in public discourse to refer to a group of cannabinoid receptor agonists. In this paper we detail the characteristics of these drugs, and present the case that the term is a misnomer. We describe the pharmacodynamics of these drugs, their epidemiology, mechanisms of action, physiological effects and how these differ substantially from delta-9-tetrahydrocannabinol (THC). We argue that not only is the term a misnomer, but it is one with negative clinical and public health implications. Rather, the substances referred to as 'synthetic cannabis' in public discourse should instead be referred to consistently as synthetic cannabinoid receptor agonists (SCRAs), a drug class distinct from plant-derived cannabinoids. SCRAs have greater potency and efficacy, and psychostimulant-like properties. While such terminology may be used in the scientific community, it is not widely used amongst the media, general public, people who use these drugs or may potentially do so. A new terminology has the potential to reduce the confusion and harms that result from the misnomer ‘synthetic cannabis’. The constant evolution of this distinct drug class necessitates a range of distinct policy responses relating to terminology, harm reduction, epidemiology, treatment, and legal status

Bayesian Inference Semantics: A Modelling System and A Test Suite

We present BIS, a Bayesian Inference Seman- tics, for probabilistic reasoning in natural lan- guage. The current system is based on the framework of Bernardy et al. (2018), but de- parts from it in important respects. BIS makes use of Bayesian learning for inferring a hy- pothesis from premises. This involves estimat- ing the probability of the hypothesis, given the data supplied by the premises of an argument. It uses a syntactic parser to generate typed syn- tactic structures that serve as input to a model generation system. Sentences are interpreted compositionally to probabilistic programs, and the corresponding truth values are estimated using sampling methods. BIS successfully deals with various probabilistic semantic phe- nomena, including frequency adverbs, gener- alised quantifiers, generics, and vague predi- cates. It performs well on a number of interest- ing probabilistic reasoning tasks. It also sus- tains most classically valid inferences (instan- tiation, de Morgan’s laws, etc.). To test BIS we have built an experimental test suite with examples of a range of probabilistic and clas- sical inference patterns

Visual coherence of moving and stationary image changes

AbstractDetection thresholds were compared for moving and stationary oscillations with equivalent contrast changes. Motion was more detectable than stationary oscillation, and the difference increased with size of the feature (a Gaussian blob). Phase discriminations between a center and two flanking features were much better for motion than for stationary oscillation. Motion phase discriminations were similar to motion detection and were robust over increases in spatial separation and temporal frequency, but not so for stationary oscillations. Separate visual motion signals were positively correlated, but visual signals for stationary oscillation were negatively correlated. Evidently, motion produces visually coherent changes in image structure, but stationary contrast oscillation does not

A Neural Model for Compositional Word Embeddings and Sentence Processing

We propose a new neural model for word embeddings, which uses Unitary Matrices as the primary device for encoding lexical information. It uses simple matrix multiplication to derive matrices for large units, yielding a sentence processing model that is strictly compositional, does not lose information over time steps, and is transparent, in the sense that word embed- dings can be analysed regardless of context. This model does not employ activation functions, and so the network is fully accessible to analysis by the methods of linear algebra at each point in its operation on an input sequence. We test it in two NLP agreement tasks and obtain rule like perfect accuracy, with greater stability than current state-of-the-art systems. Our proposed model goes some way towards offer- ing a class of computationally powerful deep learning systems that can be fully understood and compared to human cognitive processes for natural language learning and representation

GPR35 as a Novel Therapeutic Target

G protein-coupled receptors (GPCRs) remain the best studied class of cell surface receptors and the most tractable family of proteins for novel small molecule drug discovery. Despite this, a considerable number of GPCRs remain poorly characterized and in a significant number of cases, endogenous ligand(s) that activate them remain undefined or are of questionable physiological relevance. GPR35 was initially discovered over a decade ago but has remained an “orphan” receptor. Recent publications have highlighted novel ligands, both endogenously produced and synthetic, which demonstrate significant potency at this receptor. Furthermore, evidence is accumulating which highlights potential roles for GPR35 in disease and therefore, efforts to characterize GPR35 more fully and develop it as a novel therapeutic target in conditions that range from diabetes and hypertension to asthma are increasing. Recently identified ligands have shown marked species selective properties, indicating major challenges for future drug development. As we begin to understand these issues, the continuing efforts to identify novel agonist and antagonist ligands for GPR35 will help to decipher its true physiological relevance; translating multiple assay systems in vitro, to animal disease systems in vivo and finally to man

Prevalence and Characteristics Associated with Chronic Noncancer Pain in Suicide Decedents: A National Study

Objective: The aims were to estimate the prevalence of CNCP in suicide decedents, and compare sociodemographic and clinical characteristics of people who die by suicide (i) with and without a history of CNCP and (ii) among decedents with CNCP who are younger (<65 years) and older (65 + years). Method: We examined all closed cases of intentional deaths in Australia in 2014, utilizing the National Coronial Information System. Results: We identified 2,590 closed cases of intentional deaths in Australia in 2014 in decedents over 18 years of age. CNCP was identified in 14.6% of cases. Decedents with CNCP were more likely to be older, have more mental health and physical health problems, and fewer relationship problems, and were more likely to die by poisoning from drugs, compared with decedents without CNCP. Comparisons of older and younger decedents with CNCP found that compared to younger (<65 years) decedents with CNCP, older decedents (65 + years) were less likely to have mental health problems. Conclusions: This is the first national study to examine the characteristics of suicide deaths with a focus on people with CNCP. Primary care physicians should be aware of the increased risk for suicide in people living with CNCP, and it may be useful for clinicians to screen for CNCP among those presenting with suicidal behaviors

Assessing the Unitary RNN as an End-to-End Compositional Model of Syntax

We show that both an LSTM and a unitary-evolution recurrent neural network (URN) can achieve encouraging accuracy on two types of syntactic patterns: context-free long distance agreement, and mildly context-sensitive cross serial dependencies. This work extends recent experiments on deeply nested context-free long distance dependencies, with similar results. URNs differ from LSTMs in that they avoid non-linear activation functions, and they apply matrix multiplication to word embeddings encoded as unitary matrices. This permits them to retain all information in the processing of an input string over arbitrary distances. It also causes them to satisfy strict compositionality. URNs constitute a significant advance in the search for explainable models in deep learning applied to NLP