Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data

The search for effective and robust metrics has been the focus of recent theoretical and empirical work on generalization of deep neural networks (NNs). In this paper, we discuss the performance of natural language processing (NLP) models, and we evaluate various existing and novel generalization metrics. Compared to prior studies, we (i) focus on NLP instead of computer vision (CV), (ii) focus on generalization metrics that predict test error instead of the generalization gap, (iii) focus on generalization metrics that do not need the access to data, and (iv) focus on the heavy-tail (HT) phenomenon that has received comparatively less attention in the study of NNs. We extend recent HT-based work which focuses on power law (PL) distributions, and we study exponential and exponentially truncated power law (E-TPL) fitting to the empirical spectral densities (ESDs) of weight matrices. Our empirical studies are carried on (i) hundreds of Transformers trained in different settings, in which we systematically vary different hyperparameters, (ii) a total of 51 pretrained Transformers from eight families of Huggingface NLP models, including BERT, GPT2, etc., and (iii) a total of 28 existing and novel generalization metrics. From our empirical analyses, we show that shape metrics, or the metrics obtained from fitting the shape of the ESDs, perform uniformly better at predicting generalization performance than scale metrics commonly studied in the literature, as measured by the rank correlations with the generalization performance. We also show that among the three HT distributions considered in our paper, the E-TPL fitting of ESDs performs the most robustly when the models are trained in experimental settings, while the PL fitting achieves the best performance on well-trained Huggingface models, and that both E-TPL and PL metrics (which are both shape metrics) outperform scale metrics

Limits on the Stochastic Gravitational Wave Background from the North American Nanohertz Observatory for Gravitational Waves

We present an analysis of high-precision pulsar timing data taken as part of the North American Nanohertz Observatory for Gravitational waves (NANOGrav) project. We have observed 17 pulsars for a span of roughly five years using the Green Bank and Arecibo radio telescopes. We analyze these data using standard pulsar timing models, with the addition of time-variable dispersion measure and frequency-variable pulse shape terms. Sub-microsecond timing residuals are obtained in nearly all cases, and the best root-mean-square timing residuals in this set are ~30-50 ns. We present methods for analyzing post-fit timing residuals for the presence of a gravitational wave signal with a specified spectral shape. These optimally take into account the timing fluctuation power removed by the model fit, and can be applied to either data from a single pulsar, or to a set of pulsars to detect a correlated signal. We apply these methods to our dataset to set an upper limit on the strength of the nHz-frequency stochastic supermassive black hole gravitational wave background of h_c (1 yr^-1) < 7x10^-15 (95%). This result is dominated by the timing of the two best pulsars in the set, PSRs J1713+0747 and J1909-3744.Comment: To be submitted to Ap

The proangiogenic capacity of polymorphonuclear neutrophils delineated by microarray technique and by measurement of neovascularization in wounded skin of CD18-deficient mice

Growing evidence supports the concept that polymorphonuclear neutrophils (PMN) are critically involved in inflammation-mediated angiogenesis which is important for wound healing and repair. We employed an oligonucleotide microarray technique to gain further insight into the molecular mechanisms underlying the proangiogenic potential of human PMN. In addition to 18 known angiogenesis-relevant genes, we detected the expression of 10 novel genes, namely midkine, erb-B2, ets-1, transforming growth factor receptor-beta(2) and -beta(3), thrombospondin, tissue inhibitor of metalloproteinase 2, ephrin A2, ephrin B2 and restin in human PMN freshly isolated from the circulation. Gene expression was confi rmed by the RT-PCR technique. In vivo evidence for the role of PMN in neovascularization was provided by studying neovascularization in a skin model of wound healing using CD18-deficient mice which lack PMN infi ltration to sites of lesion. In CD18-deficient animals, neo- vascularization was found to be signifi cantly compromised when compared with wild- type control animals which showed profound neovascularization within the granulation tissue during the wound healing process. Thus, PMN infiltration seems to facilitate inflammation mediated angiogenesis which may be a consequence of the broad spectrum of proangiogenic factors expressed by these cells. Copyright (c) 2006 S. Karger AG, Basel

Identification of candidate tumour suppressor genes frequently methylated in renal cell carcinoma

Promoter region hyermethylation and transcriptional silencing is a frequent cause of tumour suppressor gene (TSG) inactivation in many types of human cancers. Functional epigenetic studies, in which gene expression is induced by treatment with demethylating agents, may identify novel genes with tumour-specific methylation. We used high-density gene expression microarrays in a functional epigenetic study of 11 renal cell carcinoma (RCC) cell lines. Twenty-eight genes were then selected for analysis of promoter methylation status in cell lines and primary RCC. Eight genes (BNC1, PDLIM4, RPRM, CST6, SFRP1, GREM1, COL14A1 and COL15A1) showed frequent (30% of RCC tested) tumour-specific promoter region methylation. Hypermethylation was associated with transcriptional silencing. Re-expression of BNC1, CST6, RPRM and SFRP1 suppressed the growth of RCC cell lines and RNA interference knock-down of BNC1, SFRP1 and COL14A1 increased the growth of RCC cell lines. Methylation of BNC1 or COL14A1 was associated with a poorer prognosis independent of tumour size, stage or grade. The identification of these epigenetically inactivated candidate RCC TSGs can provide insights into renal tumourigenesis and a basis for developing novel therapies and biomarkers for prognosis and detection. © 2010 Macmillan Publishers Limited.Published versio

Targeting vascular endothelial growth factor receptor 2 and protein kinase d1 related pathways by a multiple kinase inhibitor in angiogenesis and inflammation related processes in vitro.

Emerging evidence suggests that the vascular endothelial growth factor receptor 2 (VEGFR2) and protein kinase D1 (PKD1) signaling axis plays a critical role in normal and pathological angiogenesis and inflammation related processes. Despite all efforts, the currently available therapeutic interventions are limited. Prior studies have also proved that a multiple target inhibitor can be more efficient compared to a single target one. Therefore, development of novel inflammatory pathway-specific inhibitors would be of great value. To test this possibility, we screened our molecular library using recombinant kinase assays and identified the previously described compound VCC251801 with strong inhibitory effect on both VEGFR2 and PKD1. We further analyzed the effect of VCC251801 in the endothelium-derived EA.hy926 cell line and in different inflammatory cell types. In EA.hy926 cells, VCC251801 potently inhibited the intracellular activation and signaling of VEGFR2 and PKD1 which inhibition eventually resulted in diminished cell proliferation. In this model, our compound was also an efficient inhibitor of in vitro angiogenesis by interfering with endothelial cell migration and tube formation processes. Our results from functional assays in inflammatory cellular models such as neutrophils and mast cells suggested an anti-inflammatory effect of VCC251801. The neutrophil study showed that VCC251801 specifically blocked the immobilized immune-complex and the adhesion dependent TNF-alpha -fibrinogen stimulated neutrophil activation. Furthermore, similar results were found in mast cell degranulation assay where VCC251801 caused significant reduction of mast cell response. In summary, we described a novel function of a multiple kinase inhibitor which strongly inhibits the VEGFR2-PKD1 signaling and might be a novel inhibitor of pathological inflammatory pathways

Twist1 Directly Regulates Genes That Promote Cell Proliferation and Migration in Developing Heart Valves

Twist1, a basic helix-loop-helix transcription factor, is expressed in mesenchymal precursor populations during embryogenesis and in metastatic cancer cells. In the developing heart, Twist1 is highly expressed in endocardial cushion (ECC) valve mesenchymal cells and is down regulated during valve differentiation and remodeling. Previous studies demonstrated that Twist1 promotes cell proliferation, migration, and expression of primitive extracellular matrix (ECM) molecules in ECC mesenchymal cells. Furthermore, Twist1 expression is induced in human pediatric and adult diseased heart valves. However, the Twist1 downstream target genes that mediate increased cell proliferation and migration during early heart valve development remain largely unknown. Candidate gene and global gene profiling approaches were used to identify transcriptional targets of Twist1 during heart valve development. Candidate target genes were analyzed for evolutionarily conserved regions (ECRs) containing E-box consensus sequences that are potential Twist1 binding sites. ECRs containing conserved E-box sequences were identified for Twist1 responsive genes Tbx20, Cdh11, Sema3C, Rab39b, and Gadd45a. Twist1 binding to these sequences in vivo was determined by chromatin immunoprecipitation (ChIP) assays, and binding was detected in ECCs but not late stage remodeling valves. In addition identified Twist1 target genes are highly expressed in ECCs and have reduced expression during heart valve remodeling in vivo, which is consistent with the expression pattern of Twist1. Together these analyses identify multiple new genes involved in cell proliferation and migration that are differentially expressed in the developing heart valves, are responsive to Twist1 transcriptional function, and contain Twist1-responsive regulatory sequences

Mitragynine Attenuates Withdrawal Syndrome in Morphine-Withdrawn Zebrafish

A major obstacle in treating drug addiction is the severity of opiate withdrawal syndrome, which can lead to unwanted relapse. Mitragynine is the major alkaloid compound found in leaves of Mitragyna speciosa, a plant widely used by opiate addicts to mitigate the harshness of drug withdrawal. A series of experiments was conducted to investigate the effect of mitragynine on anxiety behavior, cortisol level and expression of stress pathway related genes in zebrafish undergoing morphine withdrawal phase. Adult zebrafish were subjected to two weeks chronic morphine exposure at 1.5 mg/L, followed by withdrawal for 24 hours prior to tests. Using the novel tank diving tests, we first showed that morphine-withdrawn zebrafish display anxiety-related swimming behaviors such as decreased exploratory behavior and increased erratic movement. Morphine withdrawal also elevated whole-body cortisol levels, which confirms the phenotypic stress-like behaviors. Exposing morphine-withdrawn fish to mitragynine however attenuates majority of the stress-related swimming behaviors and concomitantly lower whole-body cortisol level. Using real-time PCR gene expression analysis, we also showed that mitragynine reduces the mRNA expression of corticotropin releasing factor receptors and prodynorphin in zebrafish brain during morphine withdrawal phase, revealing for the first time a possible link between mitragynine's ability to attenuate anxiety during opiate withdrawal with the stress-related corticotropin pathway

The Extracellular Matrix and Blood Vessel Formation: Not Just a Scaffold

The extracellular matrix plays a number of important roles, among them providing structural support and information to cellular structures such as blood vessels imbedded within it. As more complex organisms have evolved, the matrix ability to direct signalling towards the vasculature and remodel in response to signalling from the vasculature has assumed progressively greater importance. This review will focus on the molecules of the extracellular matrix, specifically relating to vessel formation and their ability to signal to the surrounding cells to initiate or terminate processes involved in blood vessel formation