159 research outputs found
Introducing Foundation Models as Surrogate Models: Advancing Towards More Practical Adversarial Attacks
Recently, the no-box adversarial attack, in which the attacker lacks access
to the model's architecture, weights, and training data, become the most
practical and challenging attack setup. However, there is an unawareness of the
potential and flexibility inherent in the surrogate model selection process on
no-box setting. Inspired by the burgeoning interest in utilizing foundational
models to address downstream tasks, this paper adopts an innovative idea that
1) recasting adversarial attack as a downstream task. Specifically, image noise
generation to meet the emerging trend and 2) introducing foundational models as
surrogate models. Harnessing the concept of non-robust features, we elaborate
on two guiding principles for surrogate model selection to explain why the
foundational model is an optimal choice for this role. However, paradoxically,
we observe that these foundational models underperform. Analyzing this
unexpected behavior within the feature space, we attribute the lackluster
performance of foundational models (e.g., CLIP) to their significant
representational capacity and, conversely, their lack of discriminative
prowess. To mitigate this issue, we propose the use of a margin-based loss
strategy for the fine-tuning of foundational models on target images. The
experimental results verify that our approach, which employs the basic Fast
Gradient Sign Method (FGSM) attack algorithm, outstrips the performance of
other, more convoluted algorithms. We conclude by advocating for the research
community to consider surrogate models as crucial determinants in the
effectiveness of adversarial attacks in no-box settings. The implications of
our work bear relevance for improving the efficacy of such adversarial attacks
and the overall robustness of AI systems
Low-Mid Adversarial Perturbation against Unauthorized Face Recognition System
In light of the growing concerns regarding the unauthorized use of facial
recognition systems and its implications on individual privacy, the exploration
of adversarial perturbations as a potential countermeasure has gained traction.
However, challenges arise in effectively deploying this approach against
unauthorized facial recognition systems due to the effects of JPEG compression
on image distribution across the internet, which ultimately diminishes the
efficacy of adversarial perturbations. Existing JPEG compression-resistant
techniques struggle to strike a balance between resistance, transferability,
and attack potency. To address these limitations, we propose a novel solution
referred to as \emph{low frequency adversarial perturbation} (LFAP). This
method conditions the source model to leverage low-frequency characteristics
through adversarial training. To further enhance the performance, we introduce
an improved \emph{low-mid frequency adversarial perturbation} (LMFAP) that
incorporates mid-frequency components for an additive benefit. Our study
encompasses a range of settings to replicate genuine application scenarios,
including cross backbones, supervisory heads, training datasets, and testing
datasets. Moreover, we evaluated our approaches on a commercial black-box API,
\texttt{Face++}. The empirical results validate the cutting-edge performance
achieved by our proposed solutions.Comment: published in Information Science
Pre-training also Transfers Non-Robustness
Pre-training has enabled state-of-the-art results on many tasks. In spite of
its recognized contribution to generalization, we observed in this study that
pre-training also transfers adversarial non-robustness from pre-trained model
into fine-tuned model in the downstream tasks. Using image classification as an
example, we first conducted experiments on various datasets and network
backbones to uncover the adversarial non-robustness in fine-tuned model.
Further analysis was conducted on examining the learned knowledge of fine-tuned
model and standard model, and revealed that the reason leading to the
non-robustness is the non-robust features transferred from pre-trained model.
Finally, we analyzed the preference for feature learning of the pre-trained
model, explored the factors influencing robustness, and introduced a simple
robust pre-traning solution
Unlearnable Clusters: Towards Label-agnostic Unlearnable Examples
There is a growing interest in developing unlearnable examples (UEs) against
visual privacy leaks on the Internet. UEs are training samples added with
invisible but unlearnable noise, which have been found can prevent unauthorized
training of machine learning models. UEs typically are generated via a bilevel
optimization framework with a surrogate model to remove (minimize) errors from
the original samples, and then applied to protect the data against unknown
target models. However, existing UE generation methods all rely on an ideal
assumption called label-consistency, where the hackers and protectors are
assumed to hold the same label for a given sample. In this work, we propose and
promote a more practical label-agnostic setting, where the hackers may exploit
the protected data quite differently from the protectors. E.g., a m-class
unlearnable dataset held by the protector may be exploited by the hacker as a
n-class dataset. Existing UE generation methods are rendered ineffective in
this challenging setting. To tackle this challenge, we present a novel
technique called Unlearnable Clusters (UCs) to generate label-agnostic
unlearnable examples with cluster-wise perturbations. Furthermore, we propose
to leverage VisionandLanguage Pre-trained Models (VLPMs) like CLIP as the
surrogate model to improve the transferability of the crafted UCs to diverse
domains. We empirically verify the effectiveness of our proposed approach under
a variety of settings with different datasets, target models, and even
commercial platforms Microsoft Azure and Baidu PaddlePaddle. Code is available
at \url{https://github.com/jiamingzhang94/Unlearnable-Clusters}.Comment: CVPR202
Distribution-Aware Continual Test Time Adaptation for Semantic Segmentation
Since autonomous driving systems usually face dynamic and ever-changing
environments, continual test-time adaptation (CTTA) has been proposed as a
strategy for transferring deployed models to continually changing target
domains. However, the pursuit of long-term adaptation often introduces
catastrophic forgetting and error accumulation problems, which impede the
practical implementation of CTTA in the real world. Recently, existing CTTA
methods mainly focus on utilizing a majority of parameters to fit target domain
knowledge through self-training. Unfortunately, these approaches often amplify
the challenge of error accumulation due to noisy pseudo-labels, and pose
practical limitations stemming from the heavy computational costs associated
with entire model updates. In this paper, we propose a distribution-aware
tuning (DAT) method to make the semantic segmentation CTTA efficient and
practical in real-world applications. DAT adaptively selects and updates two
small groups of trainable parameters based on data distribution during the
continual adaptation process, including domain-specific parameters (DSP) and
task-relevant parameters (TRP). Specifically, DSP exhibits sensitivity to
outputs with substantial distribution shifts, effectively mitigating the
problem of error accumulation. In contrast, TRP are allocated to positions that
are responsive to outputs with minor distribution shifts, which are fine-tuned
to avoid the catastrophic forgetting problem. In addition, since CTTA is a
temporal task, we introduce the Parameter Accumulation Update (PAU) strategy to
collect the updated DSP and TRP in target domain sequences. We conduct
extensive experiments on two widely-used semantic segmentation CTTA benchmarks,
achieving promising performance compared to previous state-of-the-art methods
Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning
Learning Nash equilibrium (NE) in complex zero-sum games with multi-agent
reinforcement learning (MARL) can be extremely computationally expensive.
Curriculum learning is an effective way to accelerate learning, but an
under-explored dimension for generating a curriculum is the difficulty-to-learn
of the subgames -- games induced by starting from a specific state. In this
work, we present a novel subgame curriculum learning framework for zero-sum
games. It adopts an adaptive initial state distribution by resetting agents to
some previously visited states where they can quickly learn to improve
performance. Building upon this framework, we derive a subgame selection metric
that approximates the squared distance to NE values and further adopt a
particle-based state sampler for subgame generation. Integrating these
techniques leads to our new algorithm, Subgame Automatic Curriculum Learning
(SACL), which is a realization of the subgame curriculum learning framework.
SACL can be combined with any MARL algorithm such as MAPPO. Experiments in the
particle-world environment and Google Research Football environment show SACL
produces much stronger policies than baselines. In the challenging
hide-and-seek quadrant environment, SACL produces all four emergent stages and
uses only half the samples of MAPPO with self-play. The project website is at
https://sites.google.com/view/sacl-rl
Self-driven Grounding: Large Language Model Agents with Automatical Language-aligned Skill Learning
Large language models (LLMs) show their powerful automatic reasoning and
planning capability with a wealth of semantic knowledge about the human world.
However, the grounding problem still hinders the applications of LLMs in the
real-world environment. Existing studies try to fine-tune the LLM or utilize
pre-defined behavior APIs to bridge the LLMs and the environment, which not
only costs huge human efforts to customize for every single task but also
weakens the generality strengths of LLMs. To autonomously ground the LLM onto
the environment, we proposed the Self-Driven Grounding (SDG) framework to
automatically and progressively ground the LLM with self-driven skill learning.
SDG first employs the LLM to propose the hypothesis of sub-goals to achieve
tasks and then verify the feasibility of the hypothesis via interacting with
the underlying environment. Once verified, SDG can then learn generalized
skills with the guidance of these successfully grounded subgoals. These skills
can be further utilized to accomplish more complex tasks which fail to pass the
verification phase. Verified in the famous instruction following task
set-BabyAI, SDG achieves comparable performance in the most challenging tasks
compared with imitation learning methods that cost millions of demonstrations,
proving the effectiveness of learned skills and showing the feasibility and
efficiency of our framework
Online Prototype Alignment for Few-shot Policy Transfer
Domain adaptation in reinforcement learning (RL) mainly deals with the
changes of observation when transferring the policy to a new environment. Many
traditional approaches of domain adaptation in RL manage to learn a mapping
function between the source and target domain in explicit or implicit ways.
However, they typically require access to abundant data from the target domain.
Besides, they often rely on visual clues to learn the mapping function and may
fail when the source domain looks quite different from the target domain. To
address these problems, we propose a novel framework Online Prototype Alignment
(OPA) to learn the mapping function based on the functional similarity of
elements and is able to achieve the few-shot policy transfer within only
several episodes. The key insight of OPA is to introduce an exploration
mechanism that can interact with the unseen elements of the target domain in an
efficient and purposeful manner, and then connect them with the seen elements
in the source domain according to their functionalities (instead of visual
clues). Experimental results show that when the target domain looks visually
different from the source domain, OPA can achieve better transfer performance
even with much fewer samples from the target domain, outperforming prior
methods.Comment: This paper has been accepted at ICML202
Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learning
In the field of multi-task reinforcement learning, the modular principle,
which involves specializing functionalities into different modules and
combining them appropriately, has been widely adopted as a promising approach
to prevent the negative transfer problem that performance degradation due to
conflicts between tasks. However, most of the existing multi-task RL methods
only combine shared modules at the task level, ignoring that there may be
conflicts within the task. In addition, these methods do not take into account
that without constraints, some modules may learn similar functions, resulting
in restricting the model's expressiveness and generalization capability of
modular methods. In this paper, we propose the Contrastive Modules with
Temporal Attention(CMTA) method to address these limitations. CMTA constrains
the modules to be different from each other by contrastive learning and
combining shared modules at a finer granularity than the task level with
temporal attention, alleviating the negative transfer within the task and
improving the generalization ability and the performance for multi-task RL. We
conducted the experiment on Meta-World, a multi-task RL benchmark containing
various robotics manipulation tasks. Experimental results show that CMTA
outperforms learning each task individually for the first time and achieves
substantial performance improvements over the baselines.Comment: This paper has been accepted at NeurIPS 2023 as a poste
- …