Enhancing Adversarial Robustness in Low-Label Regime via Adaptively Weighted Regularization and Knowledge Distillation

Adversarial robustness is a research area that has recently received a lot of attention in the quest for trustworthy artificial intelligence. However, recent works on adversarial robustness have focused on supervised learning where it is assumed that labeled data is plentiful. In this paper, we investigate semi-supervised adversarial training where labeled data is scarce. We derive two upper bounds for the robust risk and propose a regularization term for unlabeled data motivated by these two upper bounds. Then, we develop a semi-supervised adversarial training algorithm that combines the proposed regularization term with knowledge distillation using a semi-supervised teacher (i.e., a teacher model trained using a semi-supervised learning algorithm). Our experiments show that our proposed algorithm achieves state-of-the-art performance with significant margins compared to existing algorithms. In particular, compared to supervised learning algorithms, performance of our proposed algorithm is not much worse even when the amount of labeled data is very small. For example, our algorithm with only 8\% labeled data is comparable to supervised adversarial training algorithms that use all labeled data, both in terms of standard and robust accuracies on CIFAR-10.Comment: 9 pages - Manuscript, 6 pages - Appendix, Accepted in ICCV 202

Improving Performance of Semi-Supervised Learning by Adversarial Attacks

Semi-supervised learning (SSL) algorithm is a setup built upon a realistic assumption that access to a large amount of labeled data is tough. In this study, we present a generalized framework, named SCAR, standing for Selecting Clean samples with Adversarial Robustness, for improving the performance of recent SSL algorithms. By adversarially attacking pre-trained models with semi-supervision, our framework shows substantial advances in classifying images. We introduce how adversarial attacks successfully select high-confident unlabeled data to be labeled with current predictions. On CIFAR10, three recent SSL algorithms with SCAR result in significantly improved image classification.Comment: 4 page

Offline Symbolic Analysis to Infer Total Store Order ∗

Ability to record and replay an execution can significantly help programmers debug their programs, especially parallel programs. Deterministically replaying a multiprocessor’s execution under a relaxed memory model has remained a challenging problem. This is an important problem as most modern processors only support a relaxed memory model to enable many performance critical optimizations. The most common consistency model implemented in processors is the Total Store Order (TSO). We present an efficient and low-complexity processor based solution for recording and replaying under the Total Store Order (TSO) memory model. Processor provides support for logging data fetched on cache misses. Using this information each thread can be deterministically replayed. A TSO-compliant casual order between the shared-memory accesses executed in different threads is then inferred using an offline algorithm based on Satisfiability Modulo Theory (SMT) solver. We also discuss methods to bound the search space during offline analysis and several optimizations to reduce the offline analysis time. 1

Adaptive Regularization for Adversarial Training

Adversarial training, which is to enhance robustness against adversarial attacks, has received much attention because it is easy to generate human-imperceptible perturbations of data to deceive a given deep neural network. In this paper, we propose a new adversarial training algorithm that is theoretically well motivated and empirically superior to other existing algorithms. A novel feature of the proposed algorithm is to use a data-adaptive regularization for robustifying a prediction model. We apply more regularization to data which are more vulnerable to adversarial attacks and vice versa. Even though the idea of data-adaptive regularization is not new, our data-adaptive regularization has a firm theoretical base of reducing an upper bound of the robust risk. Numerical experiments illustrate that our proposed algorithm improves the generalization (accuracy on clean samples) and robustness (accuracy on adversarial attacks) simultaneously to achieve the state-of-the-art performance

Offline Symbolic Analysis for Multi-Processor Execution Replay

Ability to replay a program’s execution on a multi-processor system can significantly help parallel programming. To replay a shared-memory multi-threaded program, existing solutions record its program input (I/O, DMA, etc.) and the shared-memory dependencies between threads. Prior processor based record-and-replay solutions are efficient, but they require non-trivial modifications to the coherency protocol and the memory sub-system for recording the shared-memory dependencies. In this paper, we propose a processor-based record-andreplay solution that does not require detecting and logging shared-memory dependencies to enable multi-processor execution replay. We show that a load-based checkpointing scheme, which was originally proposed for just recording program input, is also sufficient for replaying every thread in a multi-threaded program. Shared-memory dependencies between threads are reconstructed offline, during replay, using an algorithm based on an SMT solver. In addition to saving log space, the proposed solution significantly reduces the complexity of hardware support required for enabling replay

Deep learning alignment of bidirectional raster scanning in high speed photoacoustic microscopy

© 2022, The Author(s).Simultaneous point-by-point raster scanning of optical and acoustic beams has been widely adapted to high-speed photoacoustic microscopy (PAM) using a water-immersible microelectromechanical system or galvanometer scanner. However, when using high-speed water-immersible scanners, the two consecutively acquired bidirectional PAM images are misaligned with each other because of unstable performance, which causes a non-uniform time interval between scanning points. Therefore, only one unidirectionally acquired image is typically used; consequently, the imaging speed is reduced by half. Here, we demonstrate a scanning framework based on a deep neural network (DNN) to correct misaligned PAM images acquired via bidirectional raster scanning. The proposed method doubles the imaging speed compared to that of conventional methods by aligning nonlinear mismatched cross-sectional B-scan photoacoustic images during bidirectional raster scanning. Our DNN-assisted raster scanning framework can further potentially be applied to other raster scanning-based biomedical imaging tools, such as optical coherence tomography, ultrasound microscopy, and confocal microscopy.11Nsciescopu

Second primary malignancy risk in thyroid cancer and matched patients with and without radioiodine therapy analysis from the observational health data sciences and informatics

Purpose Risk of second primary malignancy (SPM) after radioiodine (RAI) therapy has been continuously debated. The aim of this study is to identify the risk of SPM in thyroid cancer (TC) patients with RAI compared with TC patients without RAI from matched cohort. Methods Retrospective propensity-matched cohorts were constructed across 4 hospitals in South Korea via the Observational Health Data Science and Informatics (OHDSI), and electrical health records were converted to data of common data model. TC patients who received RAI therapy constituted the target group, whereas TC patients without RAI therapy constituted the comparative group with 1:1 propensity score matching. Hazard ratio (HR) by Cox proportional hazard model was used to estimate the risk of SPM, and meta-analysis was performed to pool the HRs. Results Among a total of 24,318 patients, 5,374 patients from each group were analyzed (mean age 48.9 and 49.2, women 79.4% and 79.5% for target and comparative group, respectively). All hazard ratios of SPM in TC patients with RAI therapy were <= 1 based on 95% confidence interval(CI) from full or subgroup analyses according to thyroid cancer stage, time-at-risk period, SPM subtype (hematologic or non-hematologic), and initial age (< 30 years or >= 30 years). The HR within the target group was not significantly higher (< 1) in patients who received over 3.7 GBq of I-131 compared with patients who received less than 3.7 GBq of I-131 based on 95% CI. Conclusion There was no significant difference of the SPM risk between TC patients treated with I-131 and propensity-matched TC patients without I-131 therapy.N