On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective

ChatGPT is a recent chatbot service released by OpenAI and is receiving increasing attention over the past few months. While evaluations of various aspects of ChatGPT have been done, its robustness, i.e., the performance to unexpected inputs, is still unclear to the public. Robustness is of particular concern in responsible AI, especially for safety-critical applications. In this paper, we conduct a thorough evaluation of the robustness of ChatGPT from the adversarial and out-of-distribution (OOD) perspective. To do so, we employ the AdvGLUE and ANLI benchmarks to assess adversarial robustness and the Flipkart review and DDXPlus medical diagnosis datasets for OOD evaluation. We select several popular foundation models as baselines. Results show that ChatGPT shows consistent advantages on most adversarial and OOD classification and translation tasks. However, the absolute performance is far from perfection, which suggests that adversarial and OOD robustness remains a significant threat to foundation models. Moreover, ChatGPT shows astounding performance in understanding dialogue-related texts and we find that it tends to provide informal suggestions for medical tasks instead of definitive answers. Finally, we present in-depth discussions of possible research directions.Comment: Technical report; code is at: https://github.com/microsoft/robustlear

温度応答性蛍光リポソーム結合免疫吸着測定法の開発

早大学位記番号:新9005早稲田大

Therapeutic Evidence of Human Mesenchymal Stem Cell Transplantation for Cerebral Palsy: A Meta-Analysis of Randomized Controlled Trials

Cerebral palsy (CP) is a kind of movement and posture disorder syndrome in early childhood. In recent years, human mesenchymal stem cell (hMSC) transplantation has become a promising therapeutic strategy for CP. However, clinical evidence is still limited and controversial about clinical efficacy of hMSC therapy for CP. Our aim is to evaluate the efficacy and safety of hMSC transplantation for children with CP using a meta-analysis of randomized controlled trials (RCTs). We conducted a systematic literature search including Embase, PubMed, ClinicalTrials.gov, Cochrane Controlled Trials Register databases, Chinese Clinical Trial Registry, and Web of Science from building database to February 2020. We used Cochrane bias risk assessment for the included studies. The result of pooled analysis showed that hMSC therapy significantly increased gross motor function measure (GMFM) scores (standardized mean difference SMD=1.10, 95%CI=0.66‐1.53, P<0.00001, high-quality evidence) and comprehensive function assessment (CFA) (SMD=1.30, 95%CI=0.71‐1.90, P<0.0001, high-quality evidence) in children with CP, compared with the control group. In the subgroup analysis, the results showed that hMSC therapy significantly increased GMFM scores of 3, 6, and 12 months and CFA of 3, 6, and 12 months. Adverse event (AE) of upper respiratory infection, diarrhea, and constipation was not statistically significant between the two groups. This meta-analysis synthesized the primary outcomes and suggested that hMSC therapy is beneficial, effective, and safe in improving GMFM scores and CFA scores in children with CP. In addition, subgroup analysis showed that hMSC therapy has a lasting positive benefit for CP in 3, 6, and 12 months

Distinct Clinicopathological Features and Prognostic Values of High-, Low-, or Non-Expressing HER2 Status in Colorectal Cancer

The encouraging effects of HER2-ADC in patients with HER2-low expression cancers indicated the classical classifications based on positive and negative HER2 might no longer be suitable. However, the biology and prognosis of colorectal cancer patients with different HER2 expression status were still not clear. This is a multi-center retrospective study that included patients with histologically confirmed colorectal cancer and determined HER2 status who received radical surgical resection. HER2 immunohistochemistry (IHC) 1+ and IHC 2+ groups were combined and defined as a HER2-low group because of the concordance of clinicopathological characteristics. As compared with the HER2-high group, both the HER2-zero and the HER2-low group had less tumor with perineural invasion (14.3%, 13.1% vs. 31.6%, p = 0.001 and p p = 0.044 and p = 0.022), more RAS/BRAF mutation (52.1%, 49.9% vs. 19.5%, p p p p < 0.001). Multivariate analysis and propensity score matching also revealed that HER2-high expression was an independent prognostic factor of DFS. In conclusion, our study revealed that HER2-low colorectal cancer tumors are close to HER2-zero tumors, but different from HER2-high tumors. The routine examination of HER2 IHC is needed in early-stage colorectal cancer