Graph Inductive Biases in Transformers without Message Passing

Transformers for graph data are increasingly widely studied and successful in numerous learning tasks. Graph inductive biases are crucial for Graph Transformers, and previous works incorporate them using message-passing modules and/or positional encodings. However, Graph Transformers that use message-passing inherit known issues of message-passing, and differ significantly from Transformers used in other domains, thus making transfer of research advances more difficult. On the other hand, Graph Transformers without message-passing often perform poorly on smaller datasets, where inductive biases are more crucial. To bridge this gap, we propose the Graph Inductive bias Transformer (GRIT) -- a new Graph Transformer that incorporates graph inductive biases without using message passing. GRIT is based on several architectural changes that are each theoretically and empirically justified, including: learned relative positional encodings initialized with random walk probabilities, a flexible attention mechanism that updates node and node-pair representations, and injection of degree information in each layer. We prove that GRIT is expressive -- it can express shortest path distances and various graph propagation matrices. GRIT achieves state-of-the-art empirical performance across a variety of graph datasets, thus showing the power that Graph Transformers without message-passing can deliver.Comment: Published as a conference paper at ICML 2023; 17 page

Sample-dependent Adaptive Temperature Scaling for Improved Calibration

It is now well known that neural networks can be wrong with high confidence in their predictions, leading to poor calibration. The most common post-hoc approach to compensate for this is to perform temperature scaling, which adjusts the confidences of the predictions on any input by scaling the logits by a fixed value. Whilst this approach typically improves the average calibration across the whole test dataset, this improvement typically reduces the individual confidences of the predictions irrespective of whether the classification of a given input is correct or incorrect. With this insight, we base our method on the observation that different samples contribute to the calibration error by varying amounts, with some needing to increase their confidence and others needing to decrease it. Therefore, for each input, we propose to predict a different temperature value, allowing us to adjust the mismatch between confidence and accuracy at a finer granularity. Furthermore, we observe improved results on OOD detection and can also extract a notion of hardness for the data-points. Our method is applied post-hoc, consequently using very little computation time and with a negligible memory footprint and is applied to off-the-shelf pre-trained classifiers. We test our method on the ResNet50 and WideResNet28-10 architectures using the CIFAR10/100 and Tiny-ImageNet datasets, showing that producing per-data-point temperatures is beneficial also for the expected calibration error across the whole test set. Code is available at: https://github.com/thwjoy/adats

The Microenvironmental Effect in the Progression, Metastasis, and Dormancy of Breast Cancer: A Model System within Bone Marrow

Despite diagnostic advances, breast cancer remains the most prevalent cancer among women in the United States. The armamentarium of treatment options for metastatic disease is limited and mostly ineffective with regards to eradicating cancer. However, there have been novel findings in the recent literature that substantiate the function of the microenvironment in breast cancer progression and the support of metastasis to tertiary sites such as bone marrow. The uncovered significance of the microenvironment in the pathophysiology of breast cancer metastasis has served to challenge previously widespread theories and introduce new perspectives for the future research to eradicate breast cancer. This paper delineates the current understanding of the molecular mechanisms involved in the interactions between breast cancer cells and the microenvironment in progression, metastasis, and dormancy. The information, in addition to other mechanisms described in bone marrow, is discussed in the paper

A review of segmentation and deformable registration methods applied to adaptive cervical cancer radiation therapy treatment planning

Objective: Manual contouring and registration for radiotherapy treatment planning and online adaptation for cervical cancer radiation therapy in computed tomography (CT) and magnetic resonance images (MRI) are often necessary. However manual intervention is time consuming and may suffer from inter or intra-rater variability. In recent years a number of computer-guided automatic or semi-automatic segmentation and registration methods have been proposed. Segmentation and registration in CT and MRI for this purpose is a challenging task due to soft tissue deformation, inter-patient shape and appearance variation and anatomical changes over the course of treatment. The objective of this work is to provide a state-of-the-art review of computer-aided methods developed for adaptive treatment planning and radiation therapy planning for cervical cancer radiation therapy. Methods: Segmentation and registration methods published with the goal of cervical cancer treatment planning and adaptation have been identified from the literature (PubMed and Google Scholar). A comprehensive description of each method is provided. Similarities and differences of these methods are highlighted and the strengths and weaknesses of these methods are discussed. A discussion about choice of an appropriate method for a given modality is provided. Results: In the reviewed papers a Dice similarity coefficient of around 0.85 along with mean absolute surface distance of 2-4. mm for the clinically treated volume were reported for transfer of contours from planning day to the treatment day. Conclusions: Most segmentation and non-rigid registration methods have been primarily designed for adaptive re-planning for the transfer of contours from planning day to the treatment day. The use of shape priors significantly improved segmentation and registration accuracy compared to other models

Chemokine receptor CCR5 promotes leukocyte trafficking to the brain and survival in West Nile virus infection

The molecular immunopathogenesis of West Nile virus (WNV) infection is poorly understood. Here, we characterize a mouse model for WNV using a subcutaneous route of infection and delineate leukocyte subsets and immunoregulatory factors present in the brains of infected mice. Central nervous system (CNS) expression of the chemokine receptor CCR5 and its ligand CCL5 was prominently up-regulated by WNV, and this was associated with CNS infiltration of CD4+ and CD8+ T cells, NK1.1+ cells and macrophages expressing the receptor. The significance of CCR5 in pathogenesis was established by mortality studies in which infection of CCR5−/− mice was rapidly and uniformly fatal. In the brain, WNV-infected CCR5−/− mice had increased viral burden but markedly reduced NK1.1+ cells, macrophages, and CD4+ and CD8+ T cells compared with WNV-infected CCR5+/+ mice. Adoptive transfer of splenocytes from WNV-infected CCR5+/+ mice into infected CCR5−/− mice increased leukocyte accumulation in the CNS compared with transfer of splenocytes from infected CCR5−/− mice into infected CCR5−/− mice, and increased survival to 60%, the same as in infected CCR5+/+ control mice. We conclude that CCR5 is a critical antiviral and survival determinant in WNV infection of mice that acts by regulating trafficking of leukocytes to the infected brain

Oral Treatment with CuII(atsm) Increases Mutant SOD1 In Vivo but Protects Motor Neurons and Improves the Phenotype of a Transgenic Mouse Model of Amyotrophic Lateral Sclerosis

Mutations in the metallo-protein Cu/Zn-superoxide dismutase (SOD1) cause amyotrophic lateral sclerosis (ALS) in humans and an expression level-dependent phenotype in transgenic rodents. We show that oral treatment with the therapeutic agent diacetyl-bis(4-methylthiosemicarbazonato)copperII [CuII(atsm)] increased the concentration of mutant SOD1 (SOD1G37R) in ALS model mice, but paradoxically improved locomotor function and survival of the mice. To determine why the mice with increased levels of mutant SOD1 had an improved phenotype, we analyzed tissues by mass spectrometry. These analyses revealed most SOD1 in the spinal cord tissue of the SOD1G37R mice was Cu deficient. Treating with CuII(atsm) decreased the pool of Cu-deficient SOD1 and increased the pool of fully metallated (holo) SOD1. Tracking isotopically enriched 65CuII(atsm) confirmed the increase in holo-SOD1 involved transfer of Cu from CuII(atsm) to SOD1, suggesting the improved locomotor function and survival of the CuII(atsm)-treated SOD1G37R mice involved, at least in part, the ability of the compound to improve the Cu content of the mutant SOD1. This was supported by improved survival of SOD1G37R mice that expressed the human gene for the Cu uptake protein CTR1. Improving the metal content of mutant SOD1 in vivo with CuII(atsm) did not decrease levels of misfolded SOD1. These outcomes indicate the metal content of SOD1 may be a greater determinant of the toxicity of the protein in mutant SOD1-associated forms of ALS than the mutations themselves. Improving the metal content of SOD1 therefore represents a valid therapeutic strategy for treating ALS caused by SOD1

Non-Maxwellian Proton Velocity Distributions in Nonradiative Shocks

The Balmer line profiles of nonradiative supernova remnant shocks provide the means to measure the post-shock proton velocity distribution. While most analyses assume a Maxwellian velocity distribution, this is unlikely to be correct. In particular, neutral atoms that pass through the shock and become ionized downstream form a nonthermal distribution similar to that of pickup ions in the solar wind. We predict the H alpha line profiles from the combination of pickup protons and the ordinary shocked protons, and we consider the extent to which this distribution could affect the shock parameters derived from H alpha profiles. The Maxwellian assumption could lead to an underestimate of shock speed by up to about 15%. The isotropization of the pickup ion population generates wave energy, and we find that for the most favorable parameters this energy could significantly heat the thermal particles. Sufficiently accurate profiles could constrain the strength and direction of the magnetic field in the shocked plasma, and we discuss the distortions from a Gaussian profile to be expected in Tycho's supernova remnant.Comment: 13 pages, 6 figure

Ambient Oxygen Promotes Tumorigenesis

Oxygen serves as an essential factor for oxidative stress, and it has been shown to be a mutagen in bacteria. While it is well established that ambient oxygen can also cause genomic instability in cultured mammalian cells, its effect on de novo tumorigenesis at the organismal level is unclear. Herein, by decreasing ambient oxygen exposure, we report a ∼50% increase in the median tumor-free survival time of p53−/− mice. In the thymus, reducing oxygen exposure decreased the levels of oxidative DNA damage and RAG recombinase, both of which are known to promote lymphomagenesis in p53−/− mice. Oxygen is further shown to be associated with genomic instability in two additional cancer models involving the APC tumor suppressor gene and chemical carcinogenesis. Together, these observations represent the first report directly testing the effect of ambient oxygen on de novo tumorigenesis and provide important physiologic evidence demonstrating its critical role in increasing genomic instability in vivo

CCR5 deficiency increases risk of symptomatic West Nile virus infection

West Nile virus (WNV) is a reemerging pathogen that causes fatal encephalitis in several species, including mouse and human. Recently, we showed that the chemokine receptor CCR5 is critical for survival of mice infected with WNV, acting at the level of leukocyte trafficking to the brain. To test whether this receptor is also protective in man, we determined the frequency of CCR5Δ32, a defective CCR5 allele found predominantly in Caucasians, in two independent cohorts of patients, one from Arizona and the other from Colorado, who had laboratory-confirmed, symptomatic WNV infection. The distribution of CCR5Δ32 in a control population of healthy United States Caucasian random blood donors was in Hardy-Weinberg equilibrium and CCR5Δ32 homozygotes represented 1.0% of the total group (n = 1,318). In contrast, CCR5Δ32 homozygotes represented 4.2% of Caucasians in the Arizona cohort (odds ratios [OR] = 4.4 [95% confidence interval [CI], 1.6–11.8], P = 0.0013) and 8.3% of Caucasians in the Colorado cohort (OR = 9.1 [95% CI, 3.4–24.8], P < 0.0001). CCR5Δ32 homozygosity was significantly associated with fatal outcome in the Arizona cohort (OR = 13.2 [95% CI, 1.9–89.9], P = 0.03). We conclude that CCR5 mediates resistance to symptomatic WNV infection. Because CCR5 is also the major HIV coreceptor, these findings have important implications for the safety of CCR5-blocking agents under development for HIV/AIDS

Loss of BCAA Catabolism during Carcinogenesis Enhances mTORC1 Activity and Promotes Tumor Development and Progression

Tumors display profound changes in cellular metabolism, yet how these changes aid the development and growth of tumors is not fully understood. Here we use a multi-omic approach to examine liver carcinogenesis and regeneration, and find that progressive loss of branched-chain amino acid (BCAA) catabolism promotes tumor development and growth. In human hepatocellular carcinomas and animal models of liver cancer, suppression of BCAA catabolic enzyme expression led to BCAA accumulation in tumors, though this was not observed in regenerating liver tissues. The degree of enzyme suppression strongly correlated with tumor aggressiveness, and was an independent predictor of clinical outcome. Moreover, modulating BCAA accumulation regulated cancer cell proliferation in vitro, and tumor burden and overall survival in vivo. Dietary BCAA intake in humans also correlated with cancer mortality risk. In summary, loss of BCAA catabolism in tumors confers functional advantages, which could be exploited by therapeutic interventions in certain cancers