VDC: Versatile Data Cleanser for Detecting Dirty Samples via Visual-Linguistic Inconsistency

The role of data in building AI systems has recently been emphasized by the emerging concept of data-centric AI. Unfortunately, in the real-world, datasets may contain dirty samples, such as poisoned samples from backdoor attack, noisy labels in crowdsourcing, and even hybrids of them. The presence of such dirty samples makes the DNNs vunerable and unreliable.Hence, it is critical to detect dirty samples to improve the quality and realiability of dataset. Existing detectors only focus on detecting poisoned samples or noisy labels, that are often prone to weak generalization when dealing with dirty samples from other domains.In this paper, we find a commonality of various dirty samples is visual-linguistic inconsistency between images and associated labels. To capture the semantic inconsistency between modalities, we propose versatile data cleanser (VDC) leveraging the surpassing capabilities of multimodal large language models (MLLM) in cross-modal alignment and reasoning.It consists of three consecutive modules: the visual question generation module to generate insightful questions about the image; the visual question answering module to acquire the semantics of the visual content by answering the questions with MLLM; followed by the visual answer evaluation module to evaluate the inconsistency.Extensive experiments demonstrate its superior performance and generalization to various categories and types of dirty samples.Comment: 22 pages,5 figures,17 table

Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples

Backdoor attacks are serious security threats to machine learning models where an adversary can inject poisoned samples into the training set, causing a backdoored model which predicts poisoned samples with particular triggers to particular target classes, while behaving normally on benign samples. In this paper, we explore the task of purifying a backdoored model using a small clean dataset. By establishing the connection between backdoor risk and adversarial risk, we derive a novel upper bound for backdoor risk, which mainly captures the risk on the shared adversarial examples (SAEs) between the backdoored model and the purified model. This upper bound further suggests a novel bi-level optimization problem for mitigating backdoor using adversarial training techniques. To solve it, we propose Shared Adversarial Unlearning (SAU). Specifically, SAU first generates SAEs, and then, unlearns the generated SAEs such that they are either correctly classified by the purified model and/or differently classified by the two models, such that the backdoor effect in the backdoored model will be mitigated in the purified model. Experiments on various benchmark datasets and network architectures show that our proposed method achieves state-of-the-art performance for backdoor defense

Distraction-Aware Feature Learning for Human Attribute Recognition via Coarse-to-Fine Attention Mechanism

Recently, Human Attribute Recognition (HAR) has become a hot topic due to its scientific challenges and application potentials, where localizing attributes is a crucial stage but not well handled. In this paper, we propose a novel deep learning approach to HAR, namely Distraction-aware HAR (Da-HAR). It enhances deep CNN feature learning by improving attribute localization through a coarse-to-fine attention mechanism. At the coarse step, a self-mask block is built to roughly discriminate and reduce distractions, while at the fine step, a masked attention branch is applied to further eliminate irrelevant regions. Thanks to this mechanism, feature learning is more accurate, especially when heavy occlusions and complex backgrounds exist. Extensive experiments are conducted on the WIDER-Attribute and RAP databases, and state-of-the-art results are achieved, demonstrating the effectiveness of the proposed approach.Comment: 8 pages, 5 figures, accepted by AAAI-20 as an oral presentatio

Tunable THz Surface Plasmon Polariton based on Topological Insulator-Layered Superconductor Hybrid Structure

We theoretically investigate the surface plasmon polariton (SPP) at the interface between 3D strong topological insulator (TI) and layered superconductor-magnetic insulator structure. The tunability of SPP through electronic doping can be enhanced when the magnetic permeability of the layered structure becomes higher. When the interface is gapped by superconductivity or perpendicular magnetism, SPP dispersion is further distorted, accompanied by a shift of group velocity and penetration depth. Such a shift of SPP reaches maximum when the magnitude of Fermi level approaches the gap value, and may lead to observable effects. The tunable SPP at the interface between layered superconductor and magnetism materials in proximity to TI surface may provide new insight in the detection of Majorana Fermions.Comment: 6 pages, 4 figure

Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategy

Data-poisoning based backdoor attacks aim to insert backdoor into models by manipulating training datasets without controlling the training process of the target model. Existing attack methods mainly focus on designing triggers or fusion strategies between triggers and benign samples. However, they often randomly select samples to be poisoned, disregarding the varying importance of each poisoning sample in terms of backdoor injection. A recent selection strategy filters a fixed-size poisoning sample pool by recording forgetting events, but it fails to consider the remaining samples outside the pool from a global perspective. Moreover, computing forgetting events requires significant additional computing resources. Therefore, how to efficiently and effectively select poisoning samples from the entire dataset is an urgent problem in backdoor attacks.To address it, firstly, we introduce a poisoning mask into the regular backdoor training loss. We suppose that a backdoored model training with hard poisoning samples has a more backdoor effect on easy ones, which can be implemented by hindering the normal training process (\ie, maximizing loss \wrt mask). To further integrate it with normal training process, we then propose a learnable poisoning sample selection strategy to learn the mask together with the model parameters through a min-max optimization.Specifically, the outer loop aims to achieve the backdoor attack goal by minimizing the loss based on the selected samples, while the inner loop selects hard poisoning samples that impede this goal by maximizing the loss. After several rounds of adversarial training, we finally select effective poisoning samples with high contribution. Extensive experiments on benchmark datasets demonstrate the effectiveness and efficiency of our approach in boosting backdoor attack performance

Proximity Driven Enhanced Magnetic Order at Ferromagnetic Insulator / Magnetic Topological Insulator Interface

Magnetic exchange driven proximity effect at a magnetic insulator / topological insulator (MI/TI) interface provides a rich playground for novel phenomena as well as a way to realize low energy dissipation quantum devices. Here we report a dramatic enhancement of proximity exchange coupling in the MI / magnetic-TI EuS / Sb$_{2-x}$V$_x$Te$_3$ hybrid heterostructure, where V doping is used to drive the TI (Sb$_{2}$Te$_3$) magnetic. We observe an artificial antiferromagnetic-like structure near the MI/TI interface, which may account for the enhanced proximity coupling. The interplay between the proximity effect and doping provides insights into controllable engineering of magnetic order using a hybrid heterostructure.Comment: 5 pages, 4 figure

Dirac-Electrons-Mediated Magnetic Proximity Effect in Topological Insulator / Magnetic Insulator Heterostructures

The possible realization of dissipationless chiral edge current in a topological insulator / magnetic insulator heterostructure is based on the condition that the magnetic proximity exchange coupling at the interface is dominated by the Dirac surface states of the topological insulator. Here we report a polarized neutron reflectometry observation of Dirac electrons mediated magnetic proximity effect in a bulk-insulating topological insulator (Bi$_{0.2}$Sb$_{0.8}$)$_{2}$Te$_{3}$ / magnetic insulator EuS heterostructure. We are able to maximize the proximity induced magnetism by applying an electrical back gate to tune the Fermi level of topological insulator to be close to the charge neutral point. A phenomenological model based on diamagnetic screening is developed to explain the suppressed proximity induced magnetism at high carrier density. Our work paves the way to utilize the magnetic proximity effect at the topological insulator/magnetic insulator hetero-interface for low-power spintronic applications.Comment: 5 pages main text with 4 figures; 2 pages supplemental materials; suggestions and discussions are welcome