Imbalanced Gradients: A Subtle Cause of Overestimated Adversarial Robustness

Evaluating the robustness of a defense model is a challenging task in adversarial robustness research. Obfuscated gradients, a type of gradient masking, have previously been found to exist in many defense methods and cause a false signal of robustness. In this paper, we identify a more subtle situation called Imbalanced Gradients that can also cause overestimated adversarial robustness. The phenomenon of imbalanced gradients occurs when the gradient of one term of the margin loss dominates and pushes the attack towards to a suboptimal direction. To exploit imbalanced gradients, we formulate a Margin Decomposition (MD) attack that decomposes a margin loss into individual terms and then explores the attackability of these terms separately via a two-stage process. We also propose a MultiTargeted and an ensemble version of our MD attack. By investigating 17 defense models proposed since 2018, we find that 6 models are susceptible to imbalanced gradients and our MD attack can decrease their robustness evaluated by the best baseline standalone attack by another 2%. We also provide an in-depth analysis of the likely causes of imbalanced gradients and effective countermeasures.Comment: 19 pages, 7 figue

Eureka: Human-Level Reward Design via Coding Large Language Models

Large Language Models (LLMs) have excelled as high-level semantic planners for sequential decision-making tasks. However, harnessing them to learn complex low-level manipulation tasks, such as dexterous pen spinning, remains an open problem. We bridge this fundamental gap and present Eureka, a human-level reward design algorithm powered by LLMs. Eureka exploits the remarkable zero-shot generation, code-writing, and in-context improvement capabilities of state-of-the-art LLMs, such as GPT-4, to perform evolutionary optimization over reward code. The resulting rewards can then be used to acquire complex skills via reinforcement learning. Without any task-specific prompting or pre-defined reward templates, Eureka generates reward functions that outperform expert human-engineered rewards. In a diverse suite of 29 open-source RL environments that include 10 distinct robot morphologies, Eureka outperforms human experts on 83% of the tasks, leading to an average normalized improvement of 52%. The generality of Eureka also enables a new gradient-free in-context learning approach to reinforcement learning from human feedback (RLHF), readily incorporating human inputs to improve the quality and the safety of the generated rewards without model updating. Finally, using Eureka rewards in a curriculum learning setting, we demonstrate for the first time, a simulated Shadow Hand capable of performing pen spinning tricks, adeptly manipulating a pen in circles at rapid speed.Comment: Project website and open-source code: https://eureka-research.github.io

Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning

Augmenting pretrained language models (LMs) with a vision encoder (e.g., Flamingo) has obtained the state-of-the-art results in image-to-text generation. However, these models store all the knowledge within their parameters, thus often requiring enormous model parameters to model the abundant visual concepts and very rich textual descriptions. Additionally, they are inefficient in incorporating new data, requiring a computational-expensive fine-tuning process. In this work, we introduce a Retrieval-augmented Visual Language Model, Re-ViLM, built upon the Flamingo, that supports retrieving the relevant knowledge from the external database for zero and in-context few-shot image-to-text generations. By storing certain knowledge explicitly in the external database, our approach reduces the number of model parameters and can easily accommodate new data during evaluation by simply updating the database. We also construct an interleaved image and text data that facilitates in-context few-shot learning capabilities. We demonstrate that Re-ViLM significantly boosts performance for image-to-text generation tasks, especially for zero-shot and few-shot generation in out-of-domain settings with 4 times less parameters compared with baseline methods.Comment: Findings of EMNLP 202

Overexpressed transferrin receptor implied poor prognosis and relapse in gastrointestinal stromal tumors

Ferroptosis, as a novel-induced programmed cell death, plays critical roles in the pathogenesis of cancers. However, the promising biomarkers of ferroptosis in gastrointestinal stromal tumor (GIST) remain to be elucidated. Herein, the expression of ferroptosis-related genes was analyzed in GIST. Among the 64 ferroptosis-related genes, transferrin receptor (TFRC) expression presented a remarkable upregulation in high-risk patients through Gene Expression Omnibus (GEO) dataset analysis, as well as its significant change after imatinib was treated. Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis of TFRC-relevant genes revealed that TFRC expression was closely associated with cell growth pathways and metabolism-related pathways. Furthermore, patients at high risk of recurrence were more likely to exhibit high TFRC expression by immunohistochemistry. Additionally, high TFRC expression indicated an undesirable state of patient relapse, which could serve as a powerful significant independent predictor of recurrence-free survival (RFS). In summary, we systematically summarize the expression characteristics and clinical relevance of TFRC and show that TFRC can be used as a prognostic factor, which can be considered a potential therapeutic target in GIST

Epac1 mediates protein kinase A–independent mechanism of forskolin-activated intestinal chloride secretion

Intestinal Cl− secretion is stimulated by cyclic AMP (cAMP) and intracellular calcium ([Ca2+]i). Recent studies show that protein kinase A (PKA) and the exchange protein directly activated by cAMP (Epac) are downstream targets of cAMP. Therefore, we tested whether both PKA and Epac are involved in forskolin (FSK)/cAMP-stimulated Cl− secretion. Human intestinal T84 cells and mouse small intestine were used for short circuit current (Isc) measurement in response to agonist-stimulated Cl− secretion. FSK-stimulated Cl− secretion was completely inhibited by the additive effects of the PKA inhibitor, H89 (1 µM), and the [Ca2+]i chelator, 1,2-bis-(o-aminophenoxy)-ethane-N,N,N’,N’-tetraacetic acid, tetraacetoxymethyl ester (BAPTA-AM; 25 µM). Both FSK and the Epac activator 8-pCPT-2’-O-Me-cAMP (50 µM) elevated [Ca2+]i, activated Ras-related protein 2, and induced Cl− secretion in intact or basolateral membrane–permeabilized T84 cells and mouse ileal sheets. The effects of 8-pCPT-2’-O-Me-cAMP were completely abolished by BAPTA-AM, but not by H89. In contrast, T84 cells with silenced Epac1 had a reduced Isc response to FSK, and this response was completely inhibited by H89, but not by the phospholipase C inhibitor U73122 or BAPTA-AM. The stimulatory effect of 8-pCPT-2’-O-Me-cAMP on Cl− secretion was not abolished by cystic fibrosis transmembrane conductance (CFTR) inhibitor 172 or glibenclamide, suggesting that CFTR channels are not involved. This was confirmed by lack of effect of 8-pCPT-2’-O-Me-cAMP on whole cell patch clamp recordings of CFTR currents in Chinese hamster ovary cells transiently expressing the human CFTR channel. Furthermore, biophysical characterization of the Epac1-dependent Cl− conductance of T84 cells mounted in Ussing chambers suggested that this conductance was hyperpolarization activated, inwardly rectifying, and displayed a Cl−>Br−>I− permeability sequence. These results led us to conclude that the Epac-Rap-PLC-[Ca2+]i signaling pathway is involved in cAMP-stimulated Cl− secretion, which is carried by a novel, previously undescribed Cl− channel

Open X-Embodiment:Robotic learning datasets and RT-X models

Large, high-capacity models trained on diverse datasets have shown remarkable successes on efficiently tackling downstream applications. In domains from NLP to Computer Vision, this has led to a consolidation of pretrained models, with general pretrained backbones serving as a starting point for many applications. Can such a consolidation happen in robotics? Conventionally, robotic learning methods train a separate model for every application, every robot, and even every environment. Can we instead train "generalist" X-robot policy that can be adapted efficiently to new robots, tasks, and environments? In this paper, we provide datasets in standardized data formats and models to make it possible to explore this possibility in the context of robotic manipulation, alongside experimental results that provide an example of effective X-robot policies. We assemble a dataset from 22 different robots collected through a collaboration between 21 institutions, demonstrating 527 skills (160266 tasks). We show that a high-capacity model trained on this data, which we call RT-X, exhibits positive transfer and improves the capabilities of multiple robots by leveraging experience from other platforms. The project website is robotics-transformer-x.github.io