Why Clean Generalization and Robust Overfitting Both Happen in Adversarial Training

Adversarial training is a standard method to train deep neural networks to be robust to adversarial perturbation. Similar to surprising $\textit{clean generalization}$ ability in the standard deep learning setting, neural networks trained by adversarial training also generalize well for $\textit{unseen clean data}$. However, in constrast with clean generalization, while adversarial training method is able to achieve low $\textit{robust training error}$, there still exists a significant $\textit{robust generalization gap}$, which promotes us exploring what mechanism leads to both $\textit{clean generalization and robust overfitting (CGRO)}$ during learning process. In this paper, we provide a theoretical understanding of this CGRO phenomenon in adversarial training. First, we propose a theoretical framework of adversarial training, where we analyze $\textit{feature learning process}$ to explain how adversarial training leads network learner to CGRO regime. Specifically, we prove that, under our patch-structured dataset, the CNN model provably partially learns the true feature but exactly memorizes the spurious features from training-adversarial examples, which thus results in clean generalization and robust overfitting. For more general data assumption, we then show the efficiency of CGRO classifier from the perspective of $\textit{representation complexity}$. On the empirical side, to verify our theoretical analysis in real-world vision dataset, we investigate the $\textit{dynamics of loss landscape}$ during training. Moreover, inspired by our experiments, we prove a robust generalization bound based on $\textit{global flatness}$ of loss landscape, which may be an independent interest.Comment: 27 pages, comments welcom

Persistence In Commodity ETF Performance

Commodities ETFs have became popular investments since first introduced in the market. This type of funds provide investors a simple way to gain exposure to commodities, and these types of funds are considered as an asset class to diversify investment portfolios and as a hedge against economic recessions. With more capital invested in commodities ETFs by investors, argument about the efficiency in commodity ETF market are heated debate by economists. This paper developed a reasonable method to explore persistence in commodity ETFs. 30 commodities ETFs, which ranked high in terms of large assets, are selected during the period of 2008 to 2013. The pair-wised t-test results shows neither persistence nor reversal in commodity ETF returns for both short-term and long-term. The correlation indicates in general there exist high correlation among different ranking mix over different time frames. We conclude that there is no persistence in commodity ETF performance and the commodity ETF market is efficient

Comment on "Atomic Scale Structure and Chemical Composition across Order-Disorder Interfaces"

Interfaces have long been known to be the key to many mechanical and electric properties. To nickel base superalloys which have perfect creep and fatigue properties and have been widely used as materials of turbine blades, interfaces determine the strengthening capacities in high temperature. By means of high resolution scanning transmission electron microscopy (HRSTEM) and 3D atom probe (3DAP) tomography, Srinivasan et al. proposed a new point that in nickel base superalloys there exist two different interfacial widths across the {\gamma}/{\gamma}' interface, one corresponding to an order-disorder transition, and the other to the composition transition. We argue about this conclusion in this comment

Clinical application of transcranial magnetic stimulation for functional bowel disease

Functional bowel disorder (FBD) is a common gastrointestinal disease syndrome characterized by dysmotility and secretion without known organic lesions. The pathogenesis of FBD is still unclear. In recent years, with the rise of neurogastroenterology, it has initially revealed its close relationship with the “brain-gut axis.” Transcranial magnetic stimulation (TMS) is a technique for detecting and treating the nervous system, that is characterized by non-invasiveness and painlessness. TMS plays an important role in the diagnosis and treatment of diseases, and provides a new method for the treatment of FBD. In this paper, we summarized and analyzed the research progress of using TMS therapy applied to patients with irritable bowel syndrome and functional constipation by domestic and foreign scholars in recent years by means of literature search, and found that TMS therapy could improve the intestinal discomfort and accompanying mental symptoms in patients with FBD