Beyond Empirical Risk Minimization: Local Structure Preserving Regularization for Improving Adversarial Robustness

It is broadly known that deep neural networks are susceptible to being fooled by adversarial examples with perturbations imperceptible by humans. Various defenses have been proposed to improve adversarial robustness, among which adversarial training methods are most effective. However, most of these methods treat the training samples independently and demand a tremendous amount of samples to train a robust network, while ignoring the latent structural information among these samples. In this work, we propose a novel Local Structure Preserving (LSP) regularization, which aims to preserve the local structure of the input space in the learned embedding space. In this manner, the attacking effect of adversarial samples lying in the vicinity of clean samples can be alleviated. We show strong empirical evidence that with or without adversarial training, our method consistently improves the performance of adversarial robustness on several image classification datasets compared to the baselines and some state-of-the-art approaches, thus providing promising direction for future research.Comment: 13 pages, 4 figure

ALUM: Adversarial Data Uncertainty Modeling from Latent Model Uncertainty Compensation

It is critical that the models pay attention not only to accuracy but also to the certainty of prediction. Uncertain predictions of deep models caused by noisy data raise significant concerns in trustworthy AI areas. To explore and handle uncertainty due to intrinsic data noise, we propose a novel method called ALUM to simultaneously handle the model uncertainty and data uncertainty in a unified scheme. Rather than solely modeling data uncertainty in the ultimate layer of a deep model based on randomly selected training data, we propose to explore mined adversarial triplets to facilitate data uncertainty modeling and non-parametric uncertainty estimations to compensate for the insufficiently trained latent model layers. Thus, the critical data uncertainty and model uncertainty caused by noisy data can be readily quantified for improving model robustness. Our proposed ALUM is model-agnostic which can be easily implemented into any existing deep model with little extra computation overhead. Extensive experiments on various noisy learning tasks validate the superior robustness and generalization ability of our method. The code is released at https://github.com/wwzjer/ALUM.Comment: 10 pages, 5 figure

Dual Clustering Co-teaching with Consistent Sample Mining for Unsupervised Person Re-Identification

In unsupervised person Re-ID, peer-teaching strategy leveraging two networks to facilitate training has been proven to be an effective method to deal with the pseudo label noise. However, training two networks with a set of noisy pseudo labels reduces the complementarity of the two networks and results in label noise accumulation. To handle this issue, this paper proposes a novel Dual Clustering Co-teaching (DCCT) approach. DCCT mainly exploits the features extracted by two networks to generate two sets of pseudo labels separately by clustering with different parameters. Each network is trained with the pseudo labels generated by its peer network, which can increase the complementarity of the two networks to reduce the impact of noises. Furthermore, we propose dual clustering with dynamic parameters (DCDP) to make the network adaptive and robust to dynamically changing clustering parameters. Moreover, Consistent Sample Mining (CSM) is proposed to find the samples with unchanged pseudo labels during training for potential noisy sample removal. Extensive experiments demonstrate the effectiveness of the proposed method, which outperforms the state-of-the-art unsupervised person Re-ID methods by a considerable margin and surpasses most methods utilizing camera information

Eliciting the Translation Ability of Large Language Models via Multilingual Finetuning with Translation Instructions

Large-scale Pretrained Language Models~(LLMs), such as ChatGPT and GPT4, have shown strong abilities in multilingual translations, without being explicitly trained on parallel corpora. It is interesting how the LLMs obtain their ability to carry out translation instructions for different languages. In this paper, we present a detailed analysis by finetuning a multilingual pretrained language model, XGLM-7B, to perform multilingual translation following given instructions. Firstly, we show that the multilingual LLMs have stronger translation abilities than previously demonstrated. For a certain language pair, the performance depends on both the language families and the amount of data used in the pretraining phase. Secondly, we find that LLMs' ability to carry out translation instructions relies on the understanding of translation instruction and the alignment among different languages. With proper enhancement, LLMs could perform the translation task well even for those language pairs unseen during the instruction tuning phase

Effects of Charge Transport Materials on Blue Fluorescent Organic Light-Emitting Diodes with a Host-Dopant System

High efficiency blue fluorescent organic light-emitting diodes (OLEDs), based on 1,3-bis(carbazol-9-yl)benzene (mCP) doped with 4,4’-bis(9-ethyl-3-carbazovinylene)-1,1’-biphenyl (BCzVBi), were fabricated using four different hole transport layers (HTLs) and two different electron transport layers (ETLs). Fixing the electron transport material TPBi, four hole transport materials, including 1,1-Bis[(di-4-tolylamino)phenyl]cyclohexane (TAPC), N,N’-Di(1-naphthyl)-N,N’-diphenyl-(1,1’-biphenyl)-4’-diamine(NPB), 4,4’-Bis(N-carbazolyl)-1,1,-biphenyl (CBP) and molybdenum trioxide (MoO3), were selected to be HTLs, and the blue OLED with TAPC HTL exhibited a maximum luminance of 2955 cd/m2 and current efficiency (CE) of 5.75 cd/A at 50 mA/cm2, which are 68% and 62% higher, respectively, than those of the minimum values found in the device with MoO3 HTL. Fixing the hole transport material TAPC, the replacement of TPBi ETL with Bphen ETL can further improve the performance of the device, in which the maximum luminance can reach 3640 cd/m2 at 50 mA/cm2, which is 23% higher than that of the TPBi device. Furthermore, the lifetime of the device is also optimized by the change of ETL. These results indicate that the carrier mobility of transport materials and energy level alignment of different functional layers play important roles in the performance of the blue OLEDs. The findings suggest that selecting well-matched electron and hole transport materials is essential and beneficial for the device engineering of high-efficiency blue OLEDs

Improved Efficiency of Perovskite Light-Emitting Diodes Using a Three-Step Spin-Coated CH3NH3PbBr3 Emitter and a PEDOT:PSS/MoO3-Ammonia Composite Hole Transport Layer

High efficiency perovskite light-emitting diodes (PeLEDs) using PEDOT:PSS/MoO3-ammonia composite hole transport layers (HTLs) with different MoO3-ammonia ratios were prepared and characterized. For PeLEDs with one-step spin-coated CH3NH3PbBr3 emitter, an optimal MoO3-ammonia volume ratio (0.02) in PEDOT:PSS/MoO3-ammonia composite HTL presented a maximum luminance of 1082 cd/m2 and maximum current efficiency of 0.7 cd/A, which are 82% and 94% higher than those of the control device using pure PEDOT:PSS HTL respectively. It can be explained by that the optimized amount of MoO3-ammonia in the composite HTLs cannot only facilitate hole injection into CH3NH3PbBr3 through reducing the contact barrier, but also suppress the exciton quenching at the HTL/CH3NH3PbBr3 interface. Three-step spin coating method was further used to obtain uniform and dense CH3NH3PbBr3 films, which lead to a maximum luminance of 5044 cd/m2 and maximum current efficiency of 3.12 cd/A, showing enhancement of 750% and 767% compared with the control device respectively. The significantly improved efficiency of PeLEDs using three-step spin-coated CH3NH3PbBr3 film and an optimum PEDOT:PSS/MoO3-ammonia composite HTL can be explained by the enhanced carrier recombination through better hole injection and film morphology optimization, as well as the reduced exciton quenching at HTL/CH3NH3PbBr3 interface. These results present a promising strategy for the device engineering of high efficiency PeLEDs