Deep deterministic uncertainty: a new simple baseline

Reliable uncertainty from deterministic single-forward pass models is sought after because conventional methods of uncertainty quantification are computationally expensive. We take two complex single-forward-pass uncertainty approaches, DUQ and SNGP, and examine whether they mainly rely on a well-regularized feature space. Crucially, without using their more complex methods for estimating uncertainty, we find that a single softmax neural net with such a regularized feature-space, achieved via residual connections and spectral normalization, outperforms DUQ and SNGP's epistemic uncertainty predictions using simple Gaussian Discriminant Analysis post-training as a separate feature-space density estimator-without fine-tuning on OoD data, feature ensembling, or input pre-procressing. Our conceptually simple Deep Deterministic Uncertainty (DDU) baseline can also be used to disentangle aleatoric and epistemic uncertainty and performs as well as Deep Ensembles, the state-of-the art for uncertainty prediction, on several OoD bench-marks (CIFAR-10/100 vs SVHN/Tiny-ImageNet, ImageNet vs ImageNet-O), active learning settings across different model architectures, as well as in large scale vision tasks like semantic segmentation, while being computationally cheaper

GRAIL, an omni-directional gravitational wave detector

A cryogenic spherical and omni-directional resonant-mass detector proposed by the GRAIL collaboration is described.Comment: 5 pages, 4 figs., contribution to proceedings GW Data Analysis Workshop, Paris, nov. 199

IFNγ-stimulated B cells inhibit T follicular helper cells and protect against atherosclerosis

B and T cells are interconnected in the T follicular helper-germinal center B cell (TFH-GC B cell) axis, which is hyperactive during atherosclerosis development and loss of control along this axis results in exacerbated atherosclerosis. Inhibition of the TFH-GC B cell axis can be achieved by providing negative co-stimulation to TFH cells through the PD-1/PD-L1 pathway. Therefore, we investigated a novel therapeutic strategy using PD-L1-expressing B cells to inhibit atherosclerosis. We found that IFNγ-stimulated B cells significantly enhanced PD-L1 expression and limited TFH cell development. To determine whether IFNγ-B cells can reduce collar-induced atherosclerosis, apoE -/- mice fed a Western-type diet were treated with PBS, B cells or IFNγ-B cells for a total of 5 weeks following collar placement. IFNγ-B cells significantly increased PD-L1hi GC B cells and reduced plasmablasts. Interestingly, IFNγ-B cells-treated mice show increased atheroprotective Tregs and T cell-derived IL-10. In line with these findings, we observed a significant reduction in total lesion volume in carotid arteries of IFNγ-B cells-treated mice compared to PBS-treated mice and a similar trend was observed compared to B cell-treated mice. In conclusion, our data show that IFNγ-stimulated B cells strongly upregulate PD-L1, inhibit TFH cell responses and protect against atherosclerosis.Biopharmaceutic

Defective Autophagy in T Cells Impairs the Development of Diet-Induced Hepatic Steatosis and Atherosclerosis

Macroautophagy (or autophagy) is a conserved cellular process in which cytoplasmic cargo is targeted for lysosomal degradation. Autophagy is crucial for the functional integrity of different subsets of T cells in various developmental stages. Since atherosclerosis is an inflammatory disease of the vessel wall which is partly characterized by T cell mediated autoimmunity, we investigated how advanced atherosclerotic lesions develop in mice with T cells that lack autophagy-related protein 7 (Atg7), a protein required for functional autophagy. Mice with a T cell-specific knock-out of Atg7 (Lck-Cre Atg7f/f) had a diminished naïve CD4+ and CD8+ T cell compartment in the spleen and mediastinal lymph node as compared to littermate controls (Atg7f/f). Lck-Cre Atg7f/f and Atg7f/f mice were injected intravenously with rAAV2/8-D377Y-mPCSK9 and fed a Western-type diet to induce atherosclerosis. While Lck-Cre Atg7f/f mice had equal serum Proprotein Convertase Subtilisin/Kexin type 9 levels as compared to Atg7f/f mice, serum cholesterol levels were significantly diminished in Lck-Cre Atg7f/f mice. Histological analysis of the liver revealed less steatosis, and liver gene expression profiling showed decreased expression of genes associated with hepatic steatosis in Lck-Cre Atg7f/f mice as compared to Atg7f/f mice. The level of hepatic CD4+ and CD8+ T cells was greatly diminished but both CD4+ and CD8+ T cells showed a relative increase in their IFNγ and IL-17 production upon Atg7 deficiency. Atg7 deficiency furthermore reduced the hepatic NKT cell population which was decreased to < 0.1% of the lymphocyte population. Interestingly, T cell-specific knock-out of Atg7 decreased the mean atherosclerotic lesion size in the tri-valve area by over 50%. Taken together, T cell-specific deficiency of Atg7 resulted in a decrease in hepatic steatosis and limited inflammatory potency in the (naïve) T cell compartment in peripheral lymphoid tissues, which was associated with a strong reduction in experimental atherosclerosis

Plex: Towards Reliability using Pretrained Large Model Extensions

A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures. Probing these models' abilities in diverse ways is therefore critical to the field. In this paper, we explore the reliability of models, where we define a reliable model as one that not only achieves strong predictive performance but also performs well consistently over many decision-making tasks involving uncertainty (e.g., selective prediction, open set recognition), robust generalization (e.g., accuracy and proper scoring rules such as log-likelihood on in- and out-of-distribution datasets), and adaptation (e.g., active learning, few-shot uncertainty). We devise 10 types of tasks over 40 datasets in order to evaluate different aspects of reliability on both vision and language domains. To improve reliability, we developed ViT-Plex and T5-Plex, pretrained large model extensions for vision and language modalities, respectively. Plex greatly improves the state-of-the-art across reliability tasks, and simplifies the traditional protocol as it improves the out-of-the-box performance and does not require designing scores or tuning the model for each task. We demonstrate scaling effects over model sizes up to 1B parameters and pretraining dataset sizes up to 4B examples. We also demonstrate Plex's capabilities on challenging tasks including zero-shot open set recognition, active learning, and uncertainty in conversational language understanding.Comment: Code available at https://goo.gle/plex-cod

Diet-induced dyslipidemia induces metabolic and migratory adaptations in regulatory T cells

A hallmark of advanced atherosclerosis is inadequate immunosuppression by regulatory T(Treg) cells inside atherosclerotic lesions. Dyslipidemia has been suggested to alter Treg cellmigration by affecting the expression of specific membrane proteins, thereby decreasing Treg cellmigration towards atherosclerotic lesions. Besides membrane proteins, cellular metabolism has beenshown to be a crucial factor in Treg cell migration. We aimed to determine whether dyslipidemiacontributes to altered migration of Treg cells, in part, by affecting cellular metabolism.Dyslipidemia was induced by feeding Ldlr-/- mice a Western-type diet for 16-20weeks and intrinsic changes in Treg cells affecting their migration and metabolism were examined.Dyslipidemia was associated with altered mTORC2 signaling in Treg cells, decreased expression ofmembrane proteins involved in migration, including CD62L, CCR7 and S1Pr1, and decreased Tregcell migration towards lymph nodes. Furthermore, we discovered that diet-induced dyslipidemiainhibited mTORC1 signaling, induced PPARδ activation and increased fatty acid (FA) oxidation inTreg cells. Moreover, mass-spectrometry analysis of serum from Ldlr-/- mice with normolipidemia ordyslipidemia showed increases in multiple PPARδ ligands during dyslipidemia. Treatment with asynthetic PPARδ agonist increased the migratory capacity of Treg cells in vitro and in vivo in an FAoxidation dependent manner. Furthermore, diet-induced dyslipidemia actually enhanced Treg cellmigration into the inflamed peritoneum and into atherosclerotic lesions in vitro.Altogether, our findings implicate that dyslipidemia does not contribute toatherosclerosis by impairing Treg cell migration as dyslipidemia associated with an effector-likemigratory phenotype in Treg cells.Analytical BioScience

An Inflammatory Cascade Leading to Hyperresistinemia in Humans

BACKGROUND: Obesity, the most common cause of insulin resistance, is increasingly recognized as a low-grade inflammatory state. Adipocyte-derived resistin is a circulating protein implicated in insulin resistance in rodents, but the role of human resistin is uncertain because it is produced largely by macrophages. METHODS AND FINDINGS: The effect of endotoxin and cytokines on resistin gene and protein expression was studied in human primary blood monocytes differentiated into macrophages and in healthy human participants. Inflammatory endotoxin induced resistin in primary human macrophages via a cascade involving the secretion of inflammatory cytokines that circulate at increased levels in individuals with obesity. Induction of resistin was attenuated by drugs with dual insulin-sensitizing and anti-inflammatory properties that converge on NF-κB. In human study participants, experimental endotoxemia, which produces an insulin-resistant state, causes a dramatic rise in circulating resistin levels. Moreover, in patients with type 2 diabetes, serum resistin levels are correlated with levels of soluble tumor necrosis factor α receptor, an inflammatory marker linked to obesity, insulin resistance, and atherosclerosis. CONCLUSIONS: Inflammation is a hyperresistinemic state in humans, and cytokine induction of resistin may contribute to insulin resistance in endotoxemia, obesity, and other inflammatory states