333 research outputs found
Recommended from our members
Housing Analysis and Prediction in Melbourne Australia
Using Melbourne, Australia dataset from Kaggle.com, we analyzed the factors that determine the type of housing properties from Single-family house, Townhouse, and Apartment using multinomial logistic regression. We also predicted the property price using various machine learning algorithms such as Random Forest, Gradient Boosting, etc
COVER: A Heuristic Greedy Adversarial Attack on Prompt-based Learning in Language Models
Prompt-based learning has been proved to be an effective way in pre-trained
language models (PLMs), especially in low-resource scenarios like few-shot
settings. However, the trustworthiness of PLMs is of paramount significance and
potential vulnerabilities have been shown in prompt-based templates that could
mislead the predictions of language models, causing serious security concerns.
In this paper, we will shed light on some vulnerabilities of PLMs, by proposing
a prompt-based adversarial attack on manual templates in black box scenarios.
First of all, we design character-level and word-level heuristic approaches to
break manual templates separately. Then we present a greedy algorithm for the
attack based on the above heuristic destructive approaches. Finally, we
evaluate our approach with the classification tasks on three variants of BERT
series models and eight datasets. And comprehensive experimental results
justify the effectiveness of our approach in terms of attack success rate and
attack speed. Further experimental studies indicate that our proposed method
also displays good capabilities in scenarios with varying shot counts, template
lengths and query counts, exhibiting good generalizability
TARGET: Template-Transferable Backdoor Attack Against Prompt-based NLP Models via GPT4
Prompt-based learning has been widely applied in many low-resource NLP tasks
such as few-shot scenarios. However, this paradigm has been shown to be
vulnerable to backdoor attacks. Most of the existing attack methods focus on
inserting manually predefined templates as triggers in the pre-training phase
to train the victim model and utilize the same triggers in the downstream task
to perform inference, which tends to ignore the transferability and
stealthiness of the templates. In this work, we propose a novel approach of
TARGET (Template-trAnsfeRable backdoor attack aGainst prompt-basEd NLP models
via GPT4), which is a data-independent attack method. Specifically, we first
utilize GPT4 to reformulate manual templates to generate tone-strong and normal
templates, and the former are injected into the model as a backdoor trigger in
the pre-training phase. Then, we not only directly employ the above templates
in the downstream task, but also use GPT4 to generate templates with similar
tone to the above templates to carry out transferable attacks. Finally we have
conducted extensive experiments on five NLP datasets and three BERT series
models, with experimental results justifying that our TARGET method has better
attack performance and stealthiness compared to the two-external baseline
methods on direct attacks, and in addition achieves satisfactory attack
capability in the unseen tone-similar templates
Accelerated and Deep Expectation Maximization for One-Bit MIMO-OFDM Detection
In this paper we study the expectation maximization (EM) technique for
one-bit MIMO-OFDM detection (OMOD). Arising from the recent interest in massive
MIMO with one-bit analog-to-digital converters, OMOD is a massive-scale
problem. EM is an iterative method that can exploit the OFDM structure to
process the problem in a per-iteration efficient fashion. In this study we
analyze the convergence rate of EM for a class of approximate
maximum-likelihood OMOD formulations, or, in a broader sense, a class of
problems involving regression from quantized data. We show how the SNR and
channel conditions can have an impact on the convergence rate. We do so by
making a connection between the EM and the proximal gradient methods in the
context of OMOD. This connection also gives us insight to build new accelerated
and/or inexact EM schemes. The accelerated scheme has faster convergence in
theory, and the inexact scheme provides us with the flexibility to implement EM
more efficiently, with convergence guarantee. Furthermore we develop a deep EM
algorithm, wherein we take the structure of our inexact EM algorithm and apply
deep unfolding to train an efficient structured deep net. Simulation results
show that our accelerated exact/inexact EM algorithms run much faster than
their standard EM counterparts, and that the deep EM algorithm gives promising
detection and runtime performances
Machine Unlearning Method Based On Projection Residual
Machine learning models (mainly neural networks) are used more and more in
real life. Users feed their data to the model for training. But these processes
are often one-way. Once trained, the model remembers the data. Even when data
is removed from the dataset, the effects of these data persist in the model.
With more and more laws and regulations around the world protecting data
privacy, it becomes even more important to make models forget this data
completely through machine unlearning.
This paper adopts the projection residual method based on Newton iteration
method. The main purpose is to implement machine unlearning tasks in the
context of linear regression models and neural network models. This method
mainly uses the iterative weighting method to completely forget the data and
its corresponding influence, and its computational cost is linear in the
feature dimension of the data. This method can improve the current machine
learning method. At the same time, it is independent of the size of the
training set. Results were evaluated by feature injection testing (FIT).
Experiments show that this method is more thorough in deleting data, which is
close to model retraining.Comment: This paper is accepted by DSAA2022. The 9th IEEE International
Conference on Data Science and Advanced Analytic
FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models?
Few-shot learning aims to train models that can be generalized to novel
classes with only a few samples. Recently, a line of works are proposed to
enhance few-shot learning with accessible semantic information from class
names. However, these works focus on improving existing modules such as visual
prototypes and feature extractors of the standard few-shot learning framework.
This limits the full potential use of semantic information. In this paper, we
propose a novel few-shot learning framework that uses pre-trained language
models based on contrastive learning. To address the challenge of alignment
between visual features and textual embeddings obtained from text-based
pre-trained language model, we carefully design the textual branch of our
framework and introduce a metric module to generalize the cosine similarity.
For better transferability, we let the metric module adapt to different
few-shot tasks and adopt MAML to train the model via bi-level optimization.
Moreover, we conduct extensive experiments on multiple benchmarks to
demonstrate the effectiveness of our method
Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisition
Robotic manipulation holds the potential to replace humans in the execution
of tedious or dangerous tasks. However, control-based approaches are not
suitable due to the difficulty of formally describing open-world manipulation
in reality, and the inefficiency of existing learning methods. Thus, applying
manipulation in a wide range of scenarios presents significant challenges. In
this study, we propose a novel method for skill learning in robotic
manipulation called Tactile Active Inference Reinforcement Learning
(Tactile-AIRL), aimed at achieving efficient training. To enhance the
performance of reinforcement learning (RL), we introduce active inference,
which integrates model-based techniques and intrinsic curiosity into the RL
process. This integration improves the algorithm's training efficiency and
adaptability to sparse rewards. Additionally, we utilize a vision-based tactile
sensor to provide detailed perception for manipulation tasks. Finally, we
employ a model-based approach to imagine and plan appropriate actions through
free energy minimization. Simulation results demonstrate that our method
achieves significantly high training efficiency in non-prehensile objects
pushing tasks. It enables agents to excel in both dense and sparse reward tasks
with just a few interaction episodes, surpassing the SAC baseline. Furthermore,
we conduct physical experiments on a gripper screwing task using our method,
which showcases the algorithm's rapid learning capability and its potential for
practical applications
- …