Finding BERT’s Idiomatic Key

Sentence embeddings encode information relating to the usage of idioms in a sentence. This paper reports a set of experiments that combine a probing methodology with input masking to analyse where in a sentence this idiomatic information is taken from, and what form it takes. Our results indicate that BERT’s idiomatic key is primarily found within an idiomatic expression, but also draws on information from the surrounding context. Also, BERT can distinguish between the disruption in a sentence caused by words missing and the incongruity caused by idiomatic usage

Neuroadaptations in the Cellular and Postsynaptic Group 1 Metabotropic Glutamate Receptor mGluR5 and Homer Proteins Following Extinction of Cocaine Self-administration

This study examined the role of group1 metabotropic glutamate receptor mGluR5 and associated postsynaptic scaffolding protein Homer1b/c in behavioral plasticity after three withdrawal treatments from cocaine self-administration. Rats self-administered cocaine or saline for 14 days followed by a withdrawal period during which rats underwent extinction training, remained in their home cages, orwere placed in the self-administration chambers in the absence of extinction. Subsequently, the tissue level and distribution of proteins in the synaptosomal fraction associated with the postsynaptic densitywere examined. Cocaine self-administration followed by home cage exposure reduced the mGluR5 protein in nucleus accumbens (NA) shell and dorsolateral striatum. While extinction training reduced mGluR5 protein in NAshell, NAcore and dorsolateral striatum did not display any change. The scaffolding protein PSD95 increased in NAcore of the extinguished animals. Extinction of drug seeking was associated with a significant decrease in the synaptosomal mGluR5 protein in NAshell and an increase in dorsolateral striatum, while that of NAcore was not modified. Interestingly, both Homer1b/c and PSD95 scaffolding proteins were decreased in the synaptosomal fraction after extinction training in NAshell but not NAcore. Extinguished drug-seeking behavior was also associated with an increase in the synaptosomal actin proteins in dorsolateral striatum. Therefore, extinction of cocaine seeking is associated with neuroadaptations in mGluR5 expression and distribution that are region-specific and consist of extinction-induced reversal of cocaine-induced adaptations aswell as emergent extinction-induced alterations. Concurrent plasticity in the scaffolding proteins further suggests that mGluR5 receptor neuroadaptations may have implications for synaptic function

Glutamatergic Plasticity in Medial Prefrontal Cortex and Ventral Tegmental Area Following Extended-Access Cocaine Self-Administration

Glutamate signaling in prefrontal cortex and ventral tegmental area plays an important role in the molecular and behavioral plasticity associated with addiction to drugs of abuse. The current study investigated the expression and postsynaptic density redistribution of glutamate receptors and synaptic scaffolding proteins in dorsomedial and ventromedial prefrontal cortex and ventral tegmental area after cocaine self-administration. After 14 days of extended-access (6 h/day) cocaine self-administration, rats were exposed to one of three withdrawal regimen for 10 days. Animals either stayed in home cages (Home), returned to self-administration boxes with the levers withdrawn (Box), or underwent extinction training (Extinction). Extinction training was associated with significant glutamatergic plasticity. In dorsomedial prefrontal cortex of the Extinction group, there was an increase in postsynaptic density GluR1, PSD95, and actin proteins; while postsynaptic density mGluR5 protein decreased and there was no change in NMDAR1, Homer1b/c, or PICK1 proteins. These changes were not observed in ventromedial prefrontal cortex or ventral tegmental area. In ventral tegmental area, Extinction training reversed the decreased postsynaptic density NMDAR1 protein in the Home and Box withdrawal groups. These data suggest that extinction of drug seeking is associated with selective glutamatergic plasticity in prefrontal cortex and ventral tegmental area that include modulation of receptor trafficking to postsynaptic density

On the interaction between turbulence grids and boundary layers

Turbulence grids are widely used in wind tunnels to produce representative turbulence levels when testing aerodynamic phenomena around models. Although the purpose of the grid is to introduce a desired turbulence level in the freestream flow, the wall boundary layers of the tunnel are subjected to modification due to the presence of such grids. This could have major implications to the flow around the models to be tested and hence there is a need to further understand this interaction. The study described in this paper examines wind tunnel wall boundary layer modification by turbulence grids of different mesh sizes and porosities to understand the effect of these parameters on such interaction. Experimental results are presented in the form of pressure loss coefficients, boundary layer velocity profiles and the statistics of turbulence modification

Generative Modeling for Retinal Fundus Image Synthesis

Medical imaging datasets typically do not contain many training images, usually being deficient for training deep learning networks.We propose a deep residual variational auto-encoder and a generative adversarial network that can generate a synthetic retinal fundus image dataset with corresponding blood vessel annotation. Ourinitial experiments produce results with higher scores than the stateof the art for verifying that the structural statistics of our generatedimages are compatible with real fundus images. The successful application of generative models to generate synthetic medical datawill not only help to mitigate the small dataset problem but will alsoaddress the privacy concerns associated with medical datasets

Shapley Idioms: Analysing BERT Sentence Embeddings for General Idiom Token Identification

This article examines the basis of Natural Language Understanding of transformer based language models, such as BERT. It does this through a case study on idiom token classification. We use idiom token identification as a basis for our analysis because of the variety of information types that have previously been explored in the literature for this task, including: topic, lexical, and syntactic features. This variety of relevant information types means that the task of idiom token identification enables us to explore the forms of linguistic information that a BERT language model captures and encodes in its representations. The core of this article presents three experiments. The first experiment analyzes the effectiveness of BERT sentence embeddings for creating a general idiom token identification model and the results indicate that the BERT sentence embeddings outperform Skip-Thought. In the second and third experiment we use the game theory concept of Shapley Values to rank the usefulness of individual idiomatic expressions for model training and use this ranking to analyse the type of information that the model finds useful. We find that a combination of idiom-intrinsic and topic-based properties contribute to an expression\u27s usefulness in idiom token identification. Overall our results indicate that BERT efficiently encodes a variety of information from topic, through lexical and syntactic information. Based on these results we argue that notwithstanding recent criticisms of language model based semantics, the ability of BERT to efficiently encode a variety of linguistic information types does represent a significant step forward in natural language understanding

The Determinants of Network Growth: The Case of Commercial Online Information Networks

With the advent of modern telecommunications capabilities, networks are rapidly becoming a competitive necessity in a wide variety of industries. The literature on networks suggests that networks have characteristics (e.g., network externalities) not shared by many other products or services, and thus traditional explanations of product growth or diffusion may not apply. Because there have been few empirical studies on networks, little is known about how networks grow and compete in the marketplace and the impact of network externalities. Using commercial online information networks as a context, this research examines the determinants of network growth. We find that make effect, usability, and compatibility with the dominant technological architecture are important variables influencing network growth. The results are also weakly supportive of the network externalities hypothesis, i.e., the online services industry exhibits some extent of network externalities

NMR studies of the structure of a conserved RNA motif of 23S ribosomal RNA and its interaction with peptidyl transferase antibiotics

In this project a number of peptidyl transferase antibiotics were studied, specifically a group of aminohexose cytosine nucleoside antibiotics and their interaction with a selected number of highly conserved ribonucleic acid (RNA) motifs, designed to represent their possible binding site within the ribosome. This group of antibiotics shows a wide range of interesting properties, including antiviral and anti-tumour activity, and as they bind to a particularly conserved region in the ribosome, they are likely to be difficult for microorganisms to develop resistance to. It is hoped that once the mechanism of action of these antibiotics is better understood, that modifications to the antibiotics can be effectively made to create new or hybrid antibiotics with more selective antibacterial, or indeed antiviral or anti-tumour properties. The nuclear magnetic resonance (NMR) structure of the RNA binding, peptidyl tranferase inhibitor antibiotics amicetin, blasticidin S and gougerotin, in their native solution states, have been successfully determined. The structures all exhibit a stable conformation, stabilised by intramolecular hydrogen bonds. Amicetin was observed to be folded, distinctly different from the linear, extended conformation of amicetin previously determined by X-ray crystallography. The structure of blasticidin S was found to be very similar to its X-ray crystal structure. Gougerotin was shown to form a similar conformation to blasticidin S, save that the end chain of gougerotin was bent at right angles to the rest of the molecule, forming a structure similar to that of the major bound X-ray crystal structure of blasticidin S. All the solution structures showed a similar conformation in the analogous regions of their chemical structure, suggesting that hybrid antibiotics could be produced.Two highly conserved RNA motifs of Halobacterium halobium (H. h.) and Escherichia coli (E. coli) 23S ribosomal RNAs were chosen to investigate their interaction with amicetin. The NMR structure of the H. h. and E. coli. 29-mer RNA motifs have been determined; the motifs both form well folded A-form RNA conformations. The E. coli NMR structure differs from the X-ray crystal structure of the motif contained within the ribosome, as a highly conserved adenine residue, which resides in a bulge strongly implicated with amicetin binding, folds into the helix as opposed to being flipped out. Instead, an adjacent cytosine residue partially flips out; whereas in the crystal structure, it is folded within the helix. The NMR stuctures of the H. h. motif differs from the X-ray crystal structure of the motif, contained within the ribosome, as none of the bases are flipped out and a number of non-canonical base pairs are formed in the solution structure. To continue this study, a fully 13C and 15N isotopically labelled version of the H. h. RNA sample has been partially assigned, and an initial structure determination has been performed, using ultra high field 1 GHz spectroscopy.Addition of amicetin to both the H. h. and E. coli 29-mer RNA samples were accompanied by discrete changes to the spectra, suggesting weak interaction between the two components. These can be qualitatively interpreted to changes induced in the local conformation of the RNA motifs and the amicetin arising from the formation of a complex, between the amicetin and the bulge region of the particular motif.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Bridging Cancer Biology with the Clinic: Relative Expression of a <i>GRHL2</i>-Mediated Gene-Set Pair Predicts Breast Cancer Metastasis

Identification and characterization of crucial gene target(s) that will allow focused therapeutics development remains a challenge. We have interrogated the putative therapeutic targets associated with the transcription factor Grainy head-like 2 (GRHL2), a critical epithelial regulatory factor. We demonstrate the possibility to define the molecular functions of critical genes in terms of their personalized expression profiles, allowing appropriate functional conclusions to be derived. A novel methodology, relative expression analysis with gene-set pairs (RXA-GSP), is designed to explore the potential clinical utility of cancer-biology discovery. Observing that Grhl2-overexpression leads to increased metastatic potential in vitro, we established a model assuming Grhl2-induced or -inhibited genes confer poor or favorable prognosis respectively for cancer metastasis. Training on public gene expression profiles of 995 breast cancer patients, this method prioritized one gene-set pair (GRHL2, CDH2, FN1, CITED2, MKI67 versus CTNNB1 and CTNNA3) from all 2717 possible gene-set pairs (GSPs). The identified GSP significantly dichotomized 295 independent patients for metastasis-free survival (log-rank tested p = 0.002; severe empirical p = 0.035). It also showed evidence of clinical prognostication in another independent 388 patients collected from three studies (log-rank tested p = 3.3e–6). This GSP is independent of most traditional prognostic indicators, and is only significantly associated with the histological grade of breast cancer (p = 0.0017), a GRHL2-associated clinical character (p = 6.8e–6, Spearman correlation), suggesting that this GSP is reflective of GRHL2-mediated events. Furthermore, a literature review indicates the therapeutic potential of the identified genes. This research demonstrates a novel strategy to integrate both biological experiments and clinical gene expression profiles for extracting and elucidating the genomic impact of a novel factor, GRHL2, and its associated gene-sets on the breast cancer prognosis. Importantly, the RXA-GSP method helps to individualize breast cancer treatment. It also has the potential to contribute considerably to basic biological investigation, clinical tools, and potential therapeutic targets.</p