176 research outputs found
Towards Understanding How Self-training Tolerates Data Backdoor Poisoning
Recent studies on backdoor attacks in model training have shown that
polluting a small portion of training data is sufficient to produce incorrect
manipulated predictions on poisoned test-time data while maintaining high clean
accuracy in downstream tasks. The stealthiness of backdoor attacks has imposed
tremendous defense challenges in today's machine learning paradigm. In this
paper, we explore the potential of self-training via additional unlabeled data
for mitigating backdoor attacks. We begin by making a pilot study to show that
vanilla self-training is not effective in backdoor mitigation. Spurred by that,
we propose to defend the backdoor attacks by leveraging strong but proper data
augmentations in the self-training pseudo-labeling stage. We find that the new
self-training regime help in defending against backdoor attacks to a great
extent. Its effectiveness is demonstrated through experiments for different
backdoor triggers on CIFAR-10 and a combination of CIFAR-10 with an additional
unlabeled 500K TinyImages dataset. Finally, we explore the direction of
combining self-supervised representation learning with self-training for
further improvement in backdoor defense.Comment: Accepted at SafeAI 2023: AAAI's Workshop on Artificial Intelligence
Safet
Understanding and Improving Visual Prompting: A Label-Mapping Perspective
We revisit and advance visual prompting (VP), an input prompting technique
for vision tasks. VP can reprogram a fixed, pre-trained source model to
accomplish downstream tasks in the target domain by simply incorporating
universal prompts (in terms of input perturbation patterns) into downstream
data points. Yet, it remains elusive why VP stays effective even given a
ruleless label mapping (LM) between the source classes and the target classes.
Inspired by the above, we ask: How is LM interrelated with VP? And how to
exploit such a relationship to improve its accuracy on target tasks? We peer
into the influence of LM on VP and provide an affirmative answer that a better
'quality' of LM (assessed by mapping precision and explanation) can
consistently improve the effectiveness of VP. This is in contrast to the prior
art where the factor of LM was missing. To optimize LM, we propose a new VP
framework, termed ILM-VP (iterative label mapping-based visual prompting),
which automatically re-maps the source labels to the target labels and
progressively improves the target task accuracy of VP. Further, when using a
contrastive language-image pretrained (CLIP) model, we propose to integrate an
LM process to assist the text prompt selection of CLIP and to improve the
target task accuracy. Extensive experiments demonstrate that our proposal
significantly outperforms state-of-the-art VP methods. As highlighted below, we
show that when reprogramming an ImageNet-pretrained ResNet-18 to 13 target
tasks, our method outperforms baselines by a substantial margin, e.g., 7.9% and
6.7% accuracy improvements in transfer learning to the target Flowers102 and
CIFAR100 datasets. Besides, our proposal on CLIP-based VP provides 13.7% and
7.1% accuracy improvements on Flowers102 and DTD respectively. Our code is
available at https://github.com/OPTML-Group/ILM-VP
An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning
Recently, bi-level optimization (BLO) has taken center stage in some very
exciting developments in the area of signal processing (SP) and machine
learning (ML). Roughly speaking, BLO is a classical optimization problem that
involves two levels of hierarchy (i.e., upper and lower levels), wherein
obtaining the solution to the upper-level problem requires solving the
lower-level one. BLO has become popular largely because it is powerful in
modeling problems in SP and ML, among others, that involve optimizing nested
objective functions. Prominent applications of BLO range from resource
allocation for wireless systems to adversarial machine learning. In this work,
we focus on a class of tractable BLO problems that often appear in SP and ML
applications. We provide an overview of some basic concepts of this class of
BLO problems, such as their optimality conditions, standard algorithms
(including their optimization principles and practical implementations), as
well as how they can be leveraged to obtain state-of-the-art results for a
number of key SP and ML applications. Further, we discuss some recent advances
in BLO theory, its implications for applications, and point out some
limitations of the state-of-the-art that require significant future research
efforts. Overall, we hope that this article can serve to accelerate the
adoption of BLO as a generic tool to model, analyze, and innovate on a wide
array of emerging SP and ML applications
GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Speech Emotion Recognition
Contrastive learning based pretraining methods have recently exhibited
impressive success in diverse fields. In this paper, we propose GEmo-CLAP, a
kind of efficient gender-attribute-enhanced contrastive language-audio
pretraining (CLAP) model for speech emotion recognition. To be specific, we
first build an effective emotion CLAP model Emo-CLAP for emotion recognition,
utilizing various self-supervised learning based pre-trained models. Then,
considering the importance of the gender attribute in speech emotion modeling,
two GEmo-CLAP approaches are further proposed to integrate the emotion and
gender information of speech signals, forming more reasonable objectives.
Extensive experiments on the IEMOCAP corpus demonstrate that our proposed two
GEmo-CLAP approaches consistently outperform the baseline Emo-CLAP with
different pre-trained models, while also achieving superior recognition
performance compared with other state-of-the-art methods.Comment: 5 page
Model Sparsification Can Simplify Machine Unlearning
Recent data regulations necessitate machine unlearning (MU): The removal of
the effect of specific examples from the model. While exact unlearning is
possible by conducting a model retraining with the remaining data from scratch,
its computational cost has led to the development of approximate but efficient
unlearning schemes. Beyond data-centric MU solutions, we advance MU through a
novel model-based viewpoint: sparsification via weight pruning. Our results in
both theory and practice indicate that model sparsity can boost the
multi-criteria unlearning performance of an approximate unlearner, closing the
approximation gap, while continuing to be efficient. With this insight, we
develop two new sparsity-aware unlearning meta-schemes, termed `prune first,
then unlearn' and `sparsity-aware unlearning'. Extensive experiments show that
our findings and proposals consistently benefit MU in various scenarios,
including class-wise data scrubbing, random data scrubbing, and backdoor data
forgetting. One highlight is the 77% unlearning efficacy gain of fine-tuning
(one of the simplest approximate unlearning methods) in the proposed
sparsity-aware unlearning paradigm. Codes are available at
https://github.com/OPTML-Group/Unlearn-Sparse
Advancing Model Pruning via Bi-level Optimization
The deployment constraints in practical applications necessitate the pruning
of large-scale deep learning models, i.e., promoting their weight sparsity. As
illustrated by the Lottery Ticket Hypothesis (LTH), pruning also has the
potential of improving their generalization ability. At the core of LTH,
iterative magnitude pruning (IMP) is the predominant pruning method to
successfully find 'winning tickets'. Yet, the computation cost of IMP grows
prohibitively as the targeted pruning ratio increases. To reduce the
computation overhead, various efficient 'one-shot' pruning methods have been
developed, but these schemes are usually unable to find winning tickets as good
as IMP. This raises the question of how to close the gap between pruning
accuracy and pruning efficiency? To tackle it, we pursue the algorithmic
advancement of model pruning. Specifically, we formulate the pruning problem
from a fresh and novel viewpoint, bi-level optimization (BLO). We show that the
BLO interpretation provides a technically-grounded optimization base for an
efficient implementation of the pruning-retraining learning paradigm used in
IMP. We also show that the proposed bi-level optimization-oriented pruning
method (termed BiP) is a special class of BLO problems with a bi-linear problem
structure. By leveraging such bi-linearity, we theoretically show that BiP can
be solved as easily as first-order optimization, thus inheriting the
computation efficiency. Through extensive experiments on both structured and
unstructured pruning with 5 model architectures and 4 data sets, we demonstrate
that BiP can find better winning tickets than IMP in most cases, and is
computationally as efficient as the one-shot pruning schemes, demonstrating 2-7
times speedup over IMP for the same level of model accuracy and sparsity.Comment: Thirty-sixth Conference on Neural Information Processing Systems
(NeurIPS 2022
PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Prompts
Style voice conversion aims to transform the style of source speech to a
desired style according to real-world application demands. However, the current
style voice conversion approach relies on pre-defined labels or reference
speech to control the conversion process, which leads to limitations in style
diversity or falls short in terms of the intuitive and interpretability of
style representation. In this study, we propose PromptVC, a novel style voice
conversion approach that employs a latent diffusion model to generate a style
vector driven by natural language prompts. Specifically, the style vector is
extracted by a style encoder during training, and then the latent diffusion
model is trained independently to sample the style vector from noise, with this
process being conditioned on natural language prompts. To improve style
expressiveness, we leverage HuBERT to extract discrete tokens and replace them
with the K-Means center embedding to serve as the linguistic content, which
minimizes residual style information. Additionally, we deduplicate the same
discrete token and employ a differentiable duration predictor to re-predict the
duration of each token, which can adapt the duration of the same linguistic
content to different styles. The subjective and objective evaluation results
demonstrate the effectiveness of our proposed system.Comment: Submitted to ICASSP 202
Towards high-quality biodiesel production from microalgae using original and anaerobically-digested livestock wastewater
In this study, we conducted proof-of-concept research towards the simultaneous treatment of livestock wastewater and the generation of high-quality biodiesel, through microalgae technology. Both original (OPE) and anaerobically-digested (DPE) piggery effluents were investigated for the culture of the microalgae, Desmodesmus sp. EJ8-10. After 14 days’ cultivation, the dry biomass from microalgae cultivated in OPE increased from an initial value of 0.01 g/L to 0.33-0.39 g/L, while those growing in DPE only achieved a final dried mass of 0.15-0.35 g/L, under similar initial ammonium nitrogen (NH4+-N) concentrations. The significantly higher microalgal biomass production achieved in the OPE medium may have been supported by the abundance of both macronutrient, such as phosphorus (P), and of micronutrients, such as trace elements, present in the OPE, which may not been present in similar quantities in the DPE. However, a higher lipid content was observed (19.4-28%) in microalgal cells from DPE cultures than those (18.7-22.3%) from OPE cultures. Moreover, the fatty acid compositions in the microalgae cultured in DPE contained high levels of monounsaturated fatty acids (MUFAs) and total C16-C18 acids, which would afford a superior potential for high-quality biodiesel production. The N/P ratio (15.4:1) in OPE was much closer to that indicated by previous studies to be the most suitable (16:1) for microalgae growth, when compared with that determined from the DPE culture medium. This may facilitate protein synthesis in the algal cells and induce a lower accumulation of lipids. Based on these findings, we proposed a new flowsheet for sustainable livestock waste managemen
Critical Role of the Virus-Encoded MicroRNA-155 Ortholog in the Induction of Marek's Disease Lymphomas
Notwithstanding the well-characterised roles of a number of oncogenes in neoplastic transformation, microRNAs (miRNAs) are increasingly implicated in several human cancers. Discovery of miRNAs in several oncogenic herpesviruses such as KSHV has further highlighted the potential of virus-encoded miRNAs to contribute to their oncogenic capabilities. Nevertheless, despite the identification of several possible cancer-related genes as their targets, the direct in vivo role of virus-encoded miRNAs in neoplastic diseases such as those induced by KSHV is difficult to demonstrate in the absence of suitable models. However, excellent natural disease models of rapid-onset Marek's disease (MD) lymphomas in chickens allow examination of the oncogenic potential of virus-encoded miRNAs. Using viruses modified by reverse genetics of the infectious BAC clone of the oncogenic RB-1B strain of MDV, we show that the deletion of the six-miRNA cluster 1 from the viral genome abolished the oncogenicity of the virus. This loss of oncogenicity appeared to be primarily due to the single miRNA within the cluster, miR-M4, the ortholog of cellular miR-155, since its deletion or a 2-nucleotide mutation within its seed region was sufficient to inhibit the induction of lymphomas. The definitive role of this miR-155 ortholog in oncogenicity was further confirmed by the rescue of oncogenic phenotype by revertant viruses that expressed either the miR-M4 or the cellular homolog gga-miR-155. This is the first demonstration of the direct in vivo role of a virus-encoded miRNA in inducing tumors in a natural infection model. Furthermore, the use of viruses deleted in miRNAs as effective vaccines against virulent MDV challenge, enables the prospects of generating genetically defined attenuated vaccines
- …