14 research outputs found
Sample-efficient Adversarial Imitation Learning
Imitation learning, in which learning is performed by demonstration, has been
studied and advanced for sequential decision-making tasks in which a reward
function is not predefined. However, imitation learning methods still require
numerous expert demonstration samples to successfully imitate an expert's
behavior. To improve sample efficiency, we utilize self-supervised
representation learning, which can generate vast training signals from the
given data. In this study, we propose a self-supervised representation-based
adversarial imitation learning method to learn state and action representations
that are robust to diverse distortions and temporally predictive, on non-image
control tasks. In particular, in comparison with existing self-supervised
learning methods for tabular data, we propose a different corruption method for
state and action representations that is robust to diverse distortions. We
theoretically and empirically observe that making an informative feature
manifold with less sample complexity significantly improves the performance of
imitation learning. The proposed method shows a 39% relative improvement over
existing adversarial imitation learning methods on MuJoCo in a setting limited
to 100 expert state-action pairs. Moreover, we conduct comprehensive ablations
and additional experiments using demonstrations with varying optimality to
provide insights into a range of factors.Comment: A preliminary version of this manuscript was presented at Deep RL
Workshop, NeurIPS 202
FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networks
Personalized federated learning is aimed at allowing numerous clients to
train personalized models while participating in collaborative training in a
communication-efficient manner without exchanging private data. However, many
personalized federated learning algorithms assume that clients have the same
neural network architecture, and those for heterogeneous models remain
understudied. In this study, we propose a novel personalized federated learning
method called federated classifier averaging (FedClassAvg). Deep neural
networks for supervised learning tasks consist of feature extractor and
classifier layers. FedClassAvg aggregates classifier weights as an agreement on
decision boundaries on feature spaces so that clients with not independently
and identically distributed (non-iid) data can learn about scarce labels. In
addition, local feature representation learning is applied to stabilize the
decision boundaries and improve the local feature extraction capabilities for
clients. While the existing methods require the collection of auxiliary data or
model weights to generate a counterpart, FedClassAvg only requires clients to
communicate with a couple of fully connected layers, which is highly
communication-efficient. Moreover, FedClassAvg does not require extra
optimization problems such as knowledge transfer, which requires intensive
computation overhead. We evaluated FedClassAvg through extensive experiments
and demonstrated it outperforms the current state-of-the-art algorithms on
heterogeneous personalized federated learning tasks.Comment: Accepted to ICPP 2022. Code: https://github.com/hukla/fedclassav
Improving Visual Prompt Tuning for Self-supervised Vision Transformers
Visual Prompt Tuning (VPT) is an effective tuning method for adapting
pretrained Vision Transformers (ViTs) to downstream tasks. It leverages extra
learnable tokens, known as prompts, which steer the frozen pretrained ViTs.
Although VPT has demonstrated its applicability with supervised vision
transformers, it often underperforms with self-supervised ones. Through
empirical observations, we deduce that the effectiveness of VPT hinges largely
on the ViT blocks with which the prompt tokens interact. Specifically, VPT
shows improved performance on image classification tasks for MAE and MoCo v3
when the prompt tokens are inserted into later blocks rather than the first
block. These observations suggest that there exists an optimal location of
blocks for the insertion of prompt tokens. Unfortunately, identifying the
optimal blocks for prompts within each self-supervised ViT for diverse future
scenarios is a costly process. To mitigate this problem, we propose a simple
yet effective method that learns a gate for each ViT block to adjust its
intervention into the prompt tokens. With our method, prompt tokens are
selectively influenced by blocks that require steering for task adaptation. Our
method outperforms VPT variants in FGVC and VTAB image classification and
ADE20K semantic segmentation. The code is available at
https://github.com/ryongithub/GatedPromptTuning.Comment: International Conference on Machine Learning (ICML) 202
On the Powerfulness of Textual Outlier Exposure for Visual OoD Detection
Successful detection of Out-of-Distribution (OoD) data is becoming
increasingly important to ensure safe deployment of neural networks. One of the
main challenges in OoD detection is that neural networks output overconfident
predictions on OoD data, make it difficult to determine OoD-ness of data solely
based on their predictions. Outlier exposure addresses this issue by
introducing an additional loss that encourages low-confidence predictions on
OoD data during training. While outlier exposure has shown promising potential
in improving OoD detection performance, all previous studies on outlier
exposure have been limited to utilizing visual outliers. Drawing inspiration
from the recent advancements in vision-language pre-training, this paper
venture out to the uncharted territory of textual outlier exposure. First, we
uncover the benefits of using textual outliers by replacing real or virtual
outliers in the image-domain with textual equivalents. Then, we propose various
ways of generating preferable textual outliers. Our extensive experiments
demonstrate that generated textual outliers achieve competitive performance on
large-scale OoD and hard OoD benchmarks. Furthermore, we conduct empirical
analyses of textual outliers to provide primary criteria for designing
advantageous textual outliers: near-distribution, descriptiveness, and
inclusion of visual semantics.Comment: Accepted by NeurIPS 202
Diffusion-Stego: Training-free Diffusion Generative Steganography via Message Projection
Generative steganography is the process of hiding secret messages in
generated images instead of cover images. Existing studies on generative
steganography use GAN or Flow models to obtain high hiding message capacity and
anti-detection ability over cover images. However, they create relatively
unrealistic stego images because of the inherent limitations of generative
models. We propose Diffusion-Stego, a generative steganography approach based
on diffusion models which outperform other generative models in image
generation. Diffusion-Stego projects secret messages into latent noise of
diffusion models and generates stego images with an iterative denoising
process. Since the naive hiding of secret messages into noise boosts visual
degradation and decreases extracted message accuracy, we introduce message
projection, which hides messages into noise space while addressing these
issues. We suggest three options for message projection to adjust the trade-off
between extracted message accuracy, anti-detection ability, and image quality.
Diffusion-Stego is a training-free approach, so we can apply it to pre-trained
diffusion models which generate high-quality images, or even large-scale
text-to-image models, such as Stable diffusion. Diffusion-Stego achieved a high
capacity of messages (3.0 bpp of binary messages with 98% accuracy, and 6.0 bpp
with 90% accuracy) as well as high quality (with a FID score of 2.77 for 1.0
bpp on the FFHQ 6464 dataset) that makes it challenging to distinguish
from real images in the PNG format
Probabilistic Concept Bottleneck Models
Interpretable models are designed to make decisions in a human-interpretable
manner. Representatively, Concept Bottleneck Models (CBM) follow a two-step
process of concept prediction and class prediction based on the predicted
concepts. CBM provides explanations with high-level concepts derived from
concept predictions; thus, reliable concept predictions are important for
trustworthiness. In this study, we address the ambiguity issue that can harm
reliability. While the existence of a concept can often be ambiguous in the
data, CBM predicts concepts deterministically without considering this
ambiguity. To provide a reliable interpretation against this ambiguity, we
propose Probabilistic Concept Bottleneck Models (ProbCBM). By leveraging
probabilistic concept embeddings, ProbCBM models uncertainty in concept
prediction and provides explanations based on the concept and its corresponding
uncertainty. This uncertainty enhances the reliability of the explanations.
Furthermore, as class uncertainty is derived from concept uncertainty in
ProbCBM, we can explain class uncertainty by means of concept uncertainty. Code
is publicly available at https://github.com/ejkim47/prob-cbm.Comment: International Conference on Machine Learning (ICML) 202