Unsupervised learning of VerbNet argument structure

The relationship between a verb and the syntactic frames in which it can appear has been closely studied by psychol-ogists and linguists. Research suggests that the semantics of a verb and its arguments determine the verb’s syntactic frames,but various theories (Levin & Hovav, 2005) disagree on the nature and complexity of these relationships, in part because mostinvestigations have focused on a small subset of verbs that may not generalize. Investigating the semantic and syntactic rela-tionships present in larger sets of verbs would provide more substantial evidence for evaluating and selecting theories of verbargument structure. We report on initial analyses of the 6000+ verbs and 280+ syntactic frames of VerbNet (Kipper et al., 2008),the largest English verb syntax resource available, using nonparametric Bayesian methods (e.g. Shafto et al., 2006) for clusteranalysis and dimensionality reduction

Learning to refer informatively by amortizing pragmatic reasoning

A hallmark of human language is the ability to effectively and efficiently convey contextually relevant information. One theory for how humans reason about language is presented in the Rational Speech Acts (RSA) framework, which captures pragmatic phenomena via a process of recursive social reasoning (Goodman & Frank, 2016). However, RSA represents ideal reasoning in an unconstrained setting. We explore the idea that speakers might learn to amortize the cost of RSA computation over time by directly optimizing for successful communication with an internal listener model. In simulations with grounded neural speakers and listeners across two communication game datasets representing synthetic and human-generated data, we find that our amortized model is able to quickly generate language that is effective and concise across a range of contexts, without the need for explicit pragmatic reasoning.Comment: Accepted to CogSci 202

Learning Outside the Box: Discourse-level Features Improve Metaphor Identification.

Most current approaches to metaphor identi-fication use restricted linguistic contexts, e.g.by considering only a verb’s arguments or the sentence containing a phrase.Inspired by pragmatic accounts of metaphor, we argue that broader discourse features are crucial for bet-ter metaphor identification. We train simple gradient boosting classifiers on representations of an utterance and its surrounding discourse learned with a variety of document embedding methods, obtaining near state-of-the-art results on the 2018 VU Amsterdam metaphor iden-tification task without the complex metaphor-specific features or deep neural architectures employed by other systems.A qualitative analysis further confirms the need for broader context in metaphor processing

Improving Intrinsic Exploration with Language Abstractions

Reinforcement learning (RL) agents are particularly hard to train when rewards are sparse. One common solution is to use intrinsic rewards to encourage agents to explore their environment. However, recent intrinsic exploration methods often use state-based novelty measures which reward low-level exploration and may not scale to domains requiring more abstract skills. Instead, we explore natural language as a general medium for highlighting relevant abstractions in an environment. Unlike previous work, we evaluate whether language can improve over existing exploration methods by directly extending (and comparing to) competitive intrinsic exploration baselines: AMIGo (Campero et al., 2021) and NovelD (Zhang et al., 2021). These language-based variants outperform their non-linguistic forms by 47-85% across 13 challenging tasks from the MiniGrid and MiniHack environment suites.Comment: NeurIPS 202

Clinical significance of vasculogenic mimicry in human gliomas

Vasculogenic mimicry (VM) is known as non-endothelial tumor cell-lined microvascular channels in aggressive tumors. We have previously found the presence of VM in high-grade gliomas. In this study, we aimed to identify VM patterns in gliomas and to explore their clinical significance. Tumor samples as well as their detailed clinical/prognostic data were collected from 101 patients. Vasculogenic mimicry in the glioma samples was determined by dual staining for endothelial marker CD34 and periodic acid–Schiff (PAS). Tumor samples were also immunohistochemically stained for Ki-67, VEGF, COX-2 and MMP-9. The association between VM and the clinical characteristics of the patients were analyzed. A Kaplan–Meier survival analysis and log-rank tests were performed to compare survival times of the patients. Vasculogenic mimicry was present in 13 out of 101 samples. The higher grade gliomas had a higher incidence of VM than that of lower grade gliomas (P = 0.006). Vasculogenic mimicry channels were associated with the expression of COX-2 and MMP-9 (P < 0.05). While there was no association between the existence of VM and the sex, age and preoperative epilepsy of the patients, or expression of Ki-67 and VEGF. However, patients with VM-positive gliomas survived a shorter period of time than those with VM negative gliomas (P = 0.027). Interestingly, in high-grade gliomas, the level of microvascular density was lower in VM positive tumors than those VM negative tumors (P = 0.039). Our results suggest that VM channels in gliomas correlate with increasing malignancy and higher aggressiveness, and may provide a complementation to the tumor’s blood supply, especially in less vascularized regions, which may aid in the identification of glioma patients with a poorer prognosis

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

Humans are capable of strategically deceptive behavior: behaving helpfully in most situations, but then behaving very differently in order to pursue alternative objectives when given the opportunity. If an AI system learned such a deceptive strategy, could we detect it and remove it using current state-of-the-art safety training techniques? To study this question, we construct proof-of-concept examples of deceptive behavior in large language models (LLMs). For example, we train models that write secure code when the prompt states that the year is 2023, but insert exploitable code when the stated year is 2024. We find that such backdoor behavior can be made persistent, so that it is not removed by standard safety training techniques, including supervised fine-tuning, reinforcement learning, and adversarial training (eliciting unsafe behavior and then training to remove it). The backdoor behavior is most persistent in the largest models and in models trained to produce chain-of-thought reasoning about deceiving the training process, with the persistence remaining even when the chain-of-thought is distilled away. Furthermore, rather than removing backdoors, we find that adversarial training can teach models to better recognize their backdoor triggers, effectively hiding the unsafe behavior. Our results suggest that, once a model exhibits deceptive behavior, standard techniques could fail to remove such deception and create a false impression of safety.Comment: updated to add missing acknowledgement

Pathogenic Mechanism of the FIG4 Mutation Responsible for Charcot-Marie-Tooth Disease CMT4J

CMT4J is a severe form of Charcot-Marie-Tooth neuropathy caused by mutation of the phosphoinositide phosphatase FIG4/SAC3. Affected individuals are compound heterozygotes carrying the missense allele FIG4-I41T in combination with a null allele. Analysis using the yeast two-hybrid system demonstrated that the I41T mutation impairs interaction of FIG4 with the scaffold protein VAC14. The critical role of this interaction was confirmed by the demonstration of loss of FIG4 protein in VAC14 null mice. We developed a mouse model of CMT4J by expressing a Fig4-I41T cDNA transgene on the Fig4 null background. Expression of the mutant transcript at a level 5× higher than endogenous Fig4 completely rescued lethality, whereas 2× expression gave only partial rescue, providing a model of the human disease. The level of FIG4-I41T protein in transgenic tissues is only 2% of that predicted by the transcript level, as a consequence of the protein instability caused by impaired interaction of the mutant protein with VAC14. Analysis of patient fibroblasts demonstrated a comparably low level of mutant I41T protein. The abundance of FIG4-I41T protein in cultured cells is increased by treatment with the proteasome inhibitor MG-132. The data demonstrate that FIG4-I41T is a hypomorphic allele encoding a protein that is unstable in vivo. Expression of FIG4-I41T protein at 10% of normal level is sufficient for long-term survival, suggesting that patients with CMT4J could be treated by increased production or stabilization of the mutant protein. The transgenic model will be useful for testing in vivo interventions to increase the abundance of the mutant protein