Robustness Against Adversarial Attacks via Learning Confined Adversarial Polytopes

Deep neural networks (DNNs) could be deceived by generating human-imperceptible perturbations of clean samples. Therefore, enhancing the robustness of DNNs against adversarial attacks is a crucial task. In this paper, we aim to train robust DNNs by limiting the set of outputs reachable via a norm-bounded perturbation added to a clean sample. We refer to this set as adversarial polytope, and each clean sample has a respective adversarial polytope. Indeed, if the respective polytopes for all the samples are compact such that they do not intersect the decision boundaries of the DNN, then the DNN is robust against adversarial samples. Hence, the inner-working of our algorithm is based on learning \textbf{c}onfined \textbf{a}dversarial \textbf{p}olytopes (CAP). By conducting a thorough set of experiments, we demonstrate the effectiveness of CAP over existing adversarial robustness methods in improving the robustness of models against state-of-the-art attacks including AutoAttack.Comment: The paper has been accepted in ICASSP 202

Conditional Mutual Information Constrained Deep Learning for Classification

The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of a classification deep neural network (DNN) in the output probability distribution space of the DNN, where CMI and the ratio between CMI and NCMI represent the intra-class concentration and inter-class separation of the DNN, respectively. By using NCMI to evaluate popular DNNs pretrained over ImageNet in the literature, it is shown that their validation accuracies over ImageNet validation data set are more or less inversely proportional to their NCMI values. Based on this observation, the standard deep learning (DL) framework is further modified to minimize the standard cross entropy function subject to an NCMI constraint, yielding CMI constrained deep learning (CMIC-DL). A novel alternating learning algorithm is proposed to solve such a constrained optimization problem. Extensive experiment results show that DNNs trained within CMIC-DL outperform the state-of-the-art models trained within the standard DL and other loss functions in the literature in terms of both accuracy and robustness against adversarial attacks. In addition, visualizing the evolution of learning process through the lens of CMI and NCMI is also advocated

Collaborative decoding of critical tokens for boosting factuality of large language models

The most common training pipeline for large language models includes pretraining, finetuning and aligning phases, with their respective resulting models, such as the pretrained model and the finetuned model. Finetuned and aligned models show improved abilities of instruction following and safe generation, however their abilities to stay factual about the world are impacted by the finetuning process. Furthermore, the common practice of using sampling during generation also increases chances of hallucination. In this work, we introduce a collaborative decoding framework to harness the high factuality within pretrained models through the concept of critical tokens. We first design a critical token classifier to decide which model to use for the next token, and subsequently generates the next token using different decoding strategies. Experiments with different models and datasets show that our decoding framework is able to reduce model hallucination significantly, showcasing the importance of the collaborative decoding framework.Comment: work in progres

Channel Acquisition for HF Skywave Massive MIMO-OFDM Communications

In this paper, we investigate channel acquisition for high frequency (HF) skywave massive multiple-input multiple-output (MIMO) communications with orthogonal frequency division multiplexing (OFDM) modulation. We first introduce the concept of triple beams (TBs) in the space-frequency-time (SFT) domain and establish a TB based channel model using sampled triple steering vectors. With the established channel model, we then investigate the optimal channel estimation and pilot design for pilot segments. Specifically, we find the conditions that allow pilot reuse among multiple user terminals (UTs), which significantly reduces pilot overhead. Moreover, we propose a channel prediction method for data segments based on the estimated TB domain channel. To reduce the complexity, we are able to formulate the channel estimation as a sparse signal recovery problem due to the channel sparsity in the TB domain and then obtain the channel by the proposed constrained Bethe free energy minimization (CBFEM) based channel estimation algorithm, which can be implemented with low complexity by exploiting the structure of the TB matrix together with the chirp z-transform (CZT). Simulation results demonstrate the superior performance of the proposed channel acquisition approach.Comment: 30 pages, 4 figure

Central Aortic Systolic Blood Pressure Exhibits Advantages Over Brachial Blood Pressure Measurements in Chronic Kidney Disease Risk Prediction in Women

Background/Aims: To investigate whether the invasively obtained central aortic systolic blood pressure (CSBP) predicts chronic kidney disease (CKD) better than brachial systolic blood pressure (SBP), brachial diastolic blood pressure (DBP) and brachial pulse pressure (PP) in the middle-aged Chinese population. Methods: A cross-sectional study was carried out across China in 2009-2010 among the subjects aged 35-64 years. CSBP was measured non-invasively by radial artery applanation tonometry B-pro (A-PULSE CASP and corresponding software). CSBP, SBP, DBP and PP were standardized with Z-score and the odds ratios were calculated with multivariable logistic regression model. Results: Data of 10197 participants were analyzed. The multivariable logistic regression after adjusting for possible confounders showed that a 1-standard deviation increment in each blood pressure measurement was associated with greater risk of CKD in both men and women (P < 0.05). The association of CSBP with CKD was stronger than SBP, DBP and PP in women, while in men the association of CSBP with CKD was stronger only than PP. With CSBP and SBP entering into the multivariable logistic regression models jointly, the odds ratio (95% confidence interval) for CSBP and SBP was 1.57 (1.39-1.79) and 1.22 (1.07-1.38) in women and 1.20 (1.03-1.39) and 1.48 (1.28-1.72) in men, respectively. With CSBP and DBP entering into the multivariable logistic regression models jointly, the odds ratio (95% confidence interval) for CSBP and DBP was 1.68 (1.52-1.84) and 1.15 (1.04-1.27) in women and 1.30 (1.15-1.46) and 1.45 (1.29-1.63) in men, respectively. With CSBP and PP entering into the multivariable logistic regression models jointly, the odds ratio (95% confidence interval) for CSBP and PP was 1.75 (1.58-1.94) and 1.06 (0.96-1.17) in women and 1.58 (1.41-1.77) and 1.04 (0.93-1.17) in men, respectively. Conclusion: CSBP and brachial blood pressure measurements are all predictors of CKD, however the non-invasively obtained CSBP may offer advantages over brachial blood pressure measurements in CKD risk prediction in women

Towards Reaction Control: cis-Diastereoselective Reductive Dehydroxylation of 5-Alkyl-4-Benzyloxy-5-Hydroxy-2-Pyrrolidinones

通讯作者地址: Huang, PQA chemo-, regio-, and stereoselectively controlled reaction is highly desirable, yet challenging in organic synthesis. Diversely substituted cis and trans isomers of 2-alkyl-3-pyrrolidinols, 5-alkyl-4-hydroxy-2-pyrrolidinones, beta-hydroxy-gamma-amino acids, and their higher homologues are key structural units found in numerous drugs, drug candidates, and bioactive natural products. Previously, we established a flexible approach to trans-5-alkyl-4-benzyloxy-2-pyrrolidinones 14 and trans-6-alkyl-5-benzyloxy-2-piperidinones 15. Herein, we report a direct, flexible, moisture insensitive, and highly diastereoselective approach to the corresponding cis diastereomers 16. This stereocontrolled method is based on the MsOH-mediated (Ms=methane sulfonyl) reductive dehydroxylation of hemiaminal 12 with NaBH(OAc)(3). cis-5-Alkyl-4-benzyloxy-2-pyrrolidinones 16 are useful building blocks for the syntheses of natural products such as (+)-preussin (4) and streptopyrrolidine (5) as well as (3S,4S)-gamma-alkyl-beta-hydroxy-gamma-amino acids (6).National Basic Research Program (973 Program) of China 2010CB833200 NSF of China 20832005 21072160 Natural Science Foundation of Fujian Province 2011J0105