Enhancing Adversarial Robustness in Low-Label Regime via Adaptively Weighted Regularization and Knowledge Distillation

Adversarial robustness is a research area that has recently received a lot of attention in the quest for trustworthy artificial intelligence. However, recent works on adversarial robustness have focused on supervised learning where it is assumed that labeled data is plentiful. In this paper, we investigate semi-supervised adversarial training where labeled data is scarce. We derive two upper bounds for the robust risk and propose a regularization term for unlabeled data motivated by these two upper bounds. Then, we develop a semi-supervised adversarial training algorithm that combines the proposed regularization term with knowledge distillation using a semi-supervised teacher (i.e., a teacher model trained using a semi-supervised learning algorithm). Our experiments show that our proposed algorithm achieves state-of-the-art performance with significant margins compared to existing algorithms. In particular, compared to supervised learning algorithms, performance of our proposed algorithm is not much worse even when the amount of labeled data is very small. For example, our algorithm with only 8\% labeled data is comparable to supervised adversarial training algorithms that use all labeled data, both in terms of standard and robust accuracies on CIFAR-10.Comment: 9 pages - Manuscript, 6 pages - Appendix, Accepted in ICCV 202

Adaptive Regularization for Adversarial Training

Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention because it is easy to generate human-imperceptible perturbations of data to deceive a given deep neural network. In this paper, we propose a new adversarial training algorithm that is theoretically well motivated and empirically superior to other existing algorithms. A novel feature of the proposed algorithm is to use a data-adaptive regularization for robustifying a prediction model. We apply more regularization to data which are more vulnerable to adversarial attacks and vice versa. Even though the idea of data-adaptive regularization is not new, our data-adaptive regularization has a firm theoretical base of reducing an upper bound of the robust risk. Numerical experiments illustrate that our proposed algorithm improves the generalization (accuracy on clean samples) and robustness (accuracy on adversarial attacks) simultaneously to achieve the state-of-the-art performance

Learning fair representation with a parametric integral probability metric

As they have a vital effect on social decision-making, AI algorithms should be not only accurate but also fair. Among various algorithms for fairness AI, learning fair representation (LFR), whose goal is to find a fair representation with respect to sensitive variables such as gender and race, has received much attention. For LFR, the adversarial training scheme is popularly employed as is done in the generative adversarial network type algorithms. The choice of a discriminator, however, is done heuristically without justification. In this paper, we propose a new adversarial training scheme for LFR, where the integral probability metric (IPM) with a specific parametric family of discriminators is used. The most notable result of the proposed LFR algorithm is its theoretical guarantee about the fairness of the final prediction model, which has not been considered yet. That is, we derive theoretical relations between the fairness of representation and the fairness of the prediction model built on the top of the representation (i.e., using the representation as the input). Moreover, by numerical experiments, we show that our proposed LFR algorithm is computationally lighter and more stable, and the final prediction model is competitive or superior to other LFR algorithms using more complex discriminators.Comment: 28 pages, including references and appendi

Anti-Correlation between the Dynamics of the Active Site Loop and C-Terminal Tail in Relation to the Homodimer Asymmetry of the Mouse Erythroid 5-Aminolevulinate Synthase

Biosynthesis of heme represents a complex process that involves multiple stages controlled by different enzymes. The first of these proteins is a pyridoxal 5′-phosphate (PLP)-dependent homodimeric enzyme, 5-aminolevulinate synthase (ALAS), that catalyzes the rate-limiting step in heme biosynthesis, the condensation of glycine with succinyl-CoA. Genetic mutations in human erythroid-specific ALAS (ALAS2) are associated with two inherited blood disorders, X-linked sideroblastic anemia (XLSA) and X-linked protoporphyria (XLPP). XLSA is caused by diminished ALAS2 activity leading to decreased ALA and heme syntheses and ultimately ineffective erythropoiesis, whereas XLPP results from “gain-of-function” ALAS2 mutations and consequent overproduction of protoporphyrin IX and increase in Zn2+-protoporphyrin levels. All XLPP-linked mutations affect the intrinsically disordered C-terminal tail of ALAS2. Our earlier molecular dynamics (MD) simulation-based analysis showed that the activity of ALAS2 could be regulated by the conformational flexibility of the active site loop whose structural features and dynamics could be changed due to mutations. We also revealed that the dynamic behavior of the two protomers of the ALAS2 dimer differed. However, how the structural dynamics of ALAS2 active site loop and C-terminal tail dynamics are related to each other and contribute to the homodimer asymmetry remained unanswered questions. In this study, we used bioinformatics and computational biology tools to evaluate the role(s) of the C-terminal tail dynamics in the structure and conformational dynamics of the murine ALAS2 homodimer active site loop. To assess the structural correlation between these two regions, we analyzed their structural displacements and determined their degree of correlation. Here, we report that the dynamics of ALAS2 active site loop is anti-correlated with the dynamics of the C-terminal tail and that this anti-correlation can represent a molecular basis for the functional and dynamic asymmetry of the ALAS2 homodimer

Troponins, intrinsic disorder, and cardiomyopathy

Cardiac troponin is a dynamic complex of troponin C, troponin I, and troponin T (TnC, TnI, and TnT, respectively) found in the myocyte thin filament where it plays an essential role in cardiac muscle contraction. Mutations in troponin subunits are found in inherited cardiomyopathies, such as hypertrophic cardiomyopathy (HCM) and dilated cardiomyopathy (DCM). The highly dynamic nature of human cardiac troponin and presence of numerous flexible linkers in its subunits suggest that understanding of structural and functional properties of this important complex can benefit from the consideration of the protein intrinsic disorder phenomenon. We show here that mutations causing decrease in the disorder score in TnI and TnT are significantly more abundant in HCM and DCM than mutations leading to the increase in the disorder score. Identification and annotation of intrinsically disordered regions in each of the troponin subunits conducted in this study can help in better understanding of the roles of intrinsic disorder in regulation of interactomes and posttranslational modifications of these proteins. These observations suggest that disease-causing mutations leading to a decrease in the local flexibility of troponins can trigger a whole plethora of functional changes in the heart

Troponins, Intrinsic Disorder, and Cardiomyopathy

Cardiac troponin is a dynamic complex of troponin C, troponin I, and troponin T (TnC, TnI, and TnT, respectively) found in the myocyte thin filament where it plays an essential role in cardiac muscle contraction. Mutations in troponin subunits are found in inherited cardiomyopathies, such as hypertrophic cardiomyopathy (HCM) and dilated cardiomyopathy (DCM). The highly dynamic nature of human cardiac troponin and presence of numerous flexible linkers in its subunits suggest that understanding of structural and functional properties of this important complex can benefit from the consideration of the protein intrinsic disorder phenomenon. We show here that mutations causing decrease in the disorder score in TnI and TnT are significantly more abundant in HCM and DCM than mutations leading to the increase in the disorder score. Identification and annotation of intrinsically disordered regions in each of the troponin subunits conducted in this study can help in better understanding of the roles of intrinsic disorder in regulation of interactomes and posttranslational modifications of these proteins. These observations suggest that disease-causing mutations leading to a decrease in the local flexibility of troponins can trigger a whole plethora of functional changes in the heart

Improved resistive switching properties of solution-processed TiOx film by incorporating atomic layer deposited TiO2 layer

Resistive switching characteristics of bilayered titanium oxides layer were investigated. To improve the relatively poor electrical characteristics of solution-processed TiOx active layers, we incorporated an additional thin TiO2 (∼8 nm) layer by atomi

A putative role of the Sup35p C-terminal domain in the cytoskeleton organization during yeast mitosis

Sup35 protein (Sup35p), or eukaryotic peptide chain release factor GTP binding subunit (eRF3), is a well-known yeast prion responsible for the characteristic [PSI+] trait. N- and M-domains of this protein have been the foci of intensive research due to their importance for the prion formation. Sup35p C-terminal domain (Sup35pC) is essential for translation termination and cell viability. Deletion of Sup35pC was shown to lead to malformation of cells during mitosis. In this study we confirm that Sup35pC domain possesses high sequence and structural similarity to the eukaryotic translation elongation factor 1-α (eEF1A) from yeast and show that its sequence is conserved across different species including human. Because cell malformation during mitosis could be due to the deregulation of cytoskeleton formation, and since a Sup35 paralog eEF1A is known to act as an actin modulating protein, we focused on establishing of the relationships between the Sup35pC and modulation of the cytoskeleton formation. We found 104 co-partners between Sup35pC and EF1A of S. cerevisiae, and 18 partners of human ERF3A. Based on the analysis of known and modeled structures of some effectors and partners we found possible protein–protein interactions. Based on our study, we propose that Sup35pC may serve as actin modulator during mitosis