Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models

Studies on benign overfitting provide insights for the success of overparameterized deep learning models. In this work, we examine whether overfitting is truly benign in real-world classification tasks. We start with the observation that a ResNet model overfits benignly on Cifar10 but not benignly on ImageNet. To understand why benign overfitting fails in the ImageNet experiment, we theoretically analyze benign overfitting under a more restrictive setup where the number of parameters is not significantly larger than the number of data points. Under this mild overparameterization setup, our analysis identifies a phase change: unlike in the previous heavy overparameterization settings, benign overfitting can now fail in the presence of label noise. Our analysis explains our empirical observations, and is validated by a set of control experiments with ResNets. Our work highlights the importance of understanding implicit bias in underfitting regimes as a future direction.Comment: Published as a conference paper at ICLR 202

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization

Despite extensive studies, the underlying reason as to why overparameterized neural networks can generalize remains elusive. Existing theory shows that common stochastic optimizers prefer flatter minimizers of the training loss, and thus a natural potential explanation is that flatness implies generalization. This work critically examines this explanation. Through theoretical and empirical investigation, we identify the following three scenarios for two-layer ReLU networks: (1) flatness provably implies generalization; (2) there exist non-generalizing flattest models and sharpness minimization algorithms fail to generalize, and (3) perhaps most surprisingly, there exist non-generalizing flattest models, but sharpness minimization algorithms still generalize. Our results suggest that the relationship between sharpness and generalization subtly depends on the data distributions and the model architectures and sharpness minimization algorithms do not only minimize sharpness to achieve better generalization. This calls for the search for other explanations for the generalization of over-parameterized neural networks.Comment: 34 pages,11 figure

Practically Solving LPN in High Noise Regimes Faster Using Neural Networks

We conduct a systematic study of solving the learning parity with noise problem (LPN) using neural networks. Our main contribution is designing families of two-layer neural networks that practically outperform classical algorithms in high-noise, low-dimension regimes. We consider three settings where the numbers of LPN samples are abundant, very limited, and in between. In each setting we provide neural network models that solve LPN as fast as possible. For some settings we are also able to provide theories that explain the rationale of the design of our models. Comparing with the previous experiments of Esser, Kubler, and May (CRYPTO 2017), for dimension $n = 26$, noise rate $\tau = 0.498$, the ''Guess-then-Gaussian-elimination'' algorithm takes 3.12 days on 64 CPU cores, whereas our neural network algorithm takes 66 minutes on 8 GPUs. Our algorithm can also be plugged into the hybrid algorithms for solving middle or large dimension LPN instances.Comment: 37 page

“Reporting or Interpreting?”—A Discoursal Study of Broadcasts on NBA Games in China

From the perspective of empirical discourse analysis, this paper identifies the site broadcasters’ roles and cognitive blending process in NBA (National Basketball Association) broadcasts in China. The authors find that NBA broadcasters chiefly interpret the information they have obtained from sports sites and interviews with the coaches and players, employing various interpreting strategies, such as commentary, amplification, supplementation and restructure. Cognitively, the language that NBA broadcasters applied reveals their cognitive blending process of interpreting techniques, strategies, sports knowledge and attitudes towards the games, of who take up different roles to fulfill different communicating purposes, all of which project various cognitions on NBA games. Despite the fact that one role might make certain linguistic behaviors prevail over the others, especially their interpreting role, NBA site broadcasters coordinate it with other roles properly through which they present different levels of translational and constructional schematicity, thus yielding a coherent and constructional working mode of NBA broadcasting practice in China

Inhibition of PPARγ by BZ26, a GW9662 derivate, attenuated obesity-related breast cancer progression by inhibiting the reprogramming of mature adipocytes into to cancer associate adipocyte-like cells

Obesity has been associated with the development of 13 different types of cancers, including breast cancer. Evidence has indicated that cancer-associated adipocytes promote the proliferation, invasion, and metastasis of cancer. However, the mechanisms that link CAAs to the progression of obesity-related cancer are still unknown. Here, we found the mature adipocytes in the visceral fat of HFD-fed mice have a CAAs phenotype but the stromal vascular fraction of the visceral fat has not. Importantly, we found the derivate of the potent PPARγ antagonist GW9662, BZ26 inhibited the reprogramming of mature adipocytes in the visceral fat of HFD-fed mice into CAA-like cells and inhibited the proliferation and invasion of obesity-related breast cancer. Further study found that it mediated the browning of visceral, subcutaneous and perirenal fat and attenuated inflammation of adipose tissue and metabolic disorders. For the mechanism, we found that BZ26 bound and inhibited PPARγ by acting as a new modulator. Therefore, BZ26 serves as a novel modulator of PPARγ activity, that is, capable of inhibiting obesity-related breast cancer progression by inhibiting of CAA-like cell formation, suggesting that inhibiting the reprogramming of mature adipocytes into CAAs or CAA-like cells may be a potential therapeutic strategy for obesity-related cancer treatment

Embryonic stem cell-derived extracellular vesicles rejuvenate senescent cells and antagonize aging in mice

Aging is a degenerative process that leads to tissue dysfunction and death. Embryonic stem cells (ESCs) have great therapeutic potential for age-related diseases due to their capacity for self-renewal and plasticity. However, the use of ESCs in clinical treatment is limited by immune rejection, tumourigenicity and ethical issues. ESC-derived extracellular vesicles (EVs) may provide therapeutic effects that are comparable to those of ESCs while avoiding unwanted effects. Here, we fully evaluate the role of ESC-EVs in rejuvenation in vitro and in vivo. Using RNA sequencing (RNA-Seq) and microRNA sequencing (miRNA-Seq) screening, we found that miR-15b-5p and miR-290a-5p were highly enriched in ESC-EVs, and induced rejuvenation by silencing the Ccn2-mediated AKT/mTOR pathway. These results demonstrate that miR-15b-5p and miR-290a-5p function as potent activators of rejuvenation mediated by ESC-EVs. The rejuvenating effect of ESC-EVs was further investigated in vivo by injection into aged mice. The results showed that ESC-EVs successfully ameliorated the pathological age-related phenotypes and rescued the transcriptome profile of aged mice. Our findings demonstrate that ESC-EVs treatment can rejuvenate senescence both in vitro and in vivo and suggest the therapeutic potential of ESC-EVs as a novel cell-free alternative to ESCs for age-related diseases

A prospective phase II single-arm study and predictive factor analysis of irinotecan as third-line treatment in patients with metastatic gastric cancer

Background: Currently, there is no recommended standard third-line chemotherapy for metastatic gastric cancer. Objectives: In this study, we aimed to evaluate irinotecan’s efficacy and safety in treating metastatic gastric cancer after the failure of first- and second-line chemotherapy. Design: Prospective single-arm, two-center, phase II trial. Methods: Patients were aged 18–70 years, with histologically confirmed gastric adenocarcinoma and an Eastern Cooperative Oncology Group performance status of 0–1, progressed during or within 3 months following the last administration of second-line chemotherapy and had no other severe hematologic, cardiac, pulmonary, hepatic, or renal functional abnormalities or immunodeficiency diseases. Eligible patients received 28-day cycles of irinotecan (180 mg/m 2 intravenously, days 1 and 15) and were assessed according to the RECIST 1.1 criteria every two cycles. Patients who discontinued treatment for any reason were followed up every 2 months until death. The primary endpoint was overall survival (OS), and the secondary endpoints were progression-free survival (PFS), objective response rate (ORR), disease control rate (DCR), and toxicity. Results: A total of 98 eligible patients were enrolled in this study. In the intention-to-treat population, the median OS was 7.17 months, the median PFS was 3.47 months, and the ORR and DCR were 4.08% and 47.96%, respectively. In the per-protocol population, the median OS was 7.77 months, the median PFS was 3.47 months, and the ORR and DCR were 4.82% and 50.60%, respectively. The incidence of grade 3 or 4 hematological and non-hematological toxicities was 19.4%, and none of the patients died owing to adverse events. Cox regression analysis revealed neutropenia and baseline thrombocyte levels were independently correlated with PFS and OS. Conclusion: Irinotecan monotherapy is an efficient, well-tolerated, and economical third-line treatment for patients with metastatic gastric cancer as a third-line treatment. Trial registration: ClinicalTrials.gov identifier: NCT02662959

Fabrication of Diamond Submicron Lenses and Cylinders by ICP Etching Technique with SiO₂ Balls Mask

Submicron lenses and cylinders exhibiting excellent properties in photodetector and quantum applications have been fabricated on a diamond surface by an inductively-coupled plasma (ICP) etching technique. During ICP etching, a layer containing 500 nm diameter balls of SiO2 was employed as mask. By changing the mixing ratio of O2, Ar and CF4 during ICP etching, several submicron structures were fabricated, such as cylinders and lenses. The simulation results demonstrated that such submicron structures on a diamond’s surface can greatly enhance the photon out-coupling efficiency of embedded nitrogen-vacancy center

Crystal Co_<i>x</i>B (<i>x</i> = 1–3) Synthesized by a Ball-Milling Method as High-Performance Electrocatalysts for the Oxygen Evolution Reaction

Development of noble-metal-free and active electrocatalysts is crucial for the oxygen evolution reaction (OER) in the water-splitting process. Herein, crystal Co_<i>x</i>B catalysts (<i>x</i> = 1–3) of the OER are fabricated by a ball-milling method. Among these Co_<i>x</i>B catalysts, Co₂B exhibits the best OER activity, with a current density of 10 mA cm^–2 at an overpotential of 287 mV in 1 M KOH solution. Such OER activity of Co₂B is favorably comparable to that of the commercial IrO₂ and most recently reported OER catalysts. Furthermore, the Co₂B catalyst exhibits excellent stability with a stable current density of 50 mA cm^–2 over 12 h of continuous electrolysis operation. X-ray photoelectron spectroscopy and cyclic voltammetry demonstrate that the B in Co_<i>x</i>B makes oxidation easier, leading to their enhanced OER activities in comparison to metal Co. In addition, the Co₂B electrocatalyst also exhibits high activity in the hydrogen evolution reaction; thus, the catalyst can be used as a bifunctional catalyst for full water splitting