MIP: CLIP-based Image Reconstruction from PEFT Gradients

Contrastive Language-Image Pre-training (CLIP) model, as an effective pre-trained multimodal neural network, has been widely used in distributed machine learning tasks, especially Federated Learning (FL). Typically, CLIP-based FL adopts Parameter-Efficient Fine-Tuning (PEFT) for model training, which only fine-tunes adapter parameters or soft prompts rather than the full parameters. Although PEFT is different from the traditional training mode, in this paper, we theoretically analyze that the gradients of adapters or soft prompts can still be used to perform image reconstruction attacks. Based on our theoretical analysis, we propose Multm-In-Parvo (MIP), a proprietary reconstruction attack method targeting CLIP-based distributed machine learning architecture. Specifically, MIP can reconstruct CLIP training images according to the gradients of soft prompts or an adapter. In addition, MIP includes a label prediction strategy to accelerate convergence and an inverse gradient estimation mechanism to avoid the vanishing gradient problem on the text encoder. Experimental results show that MIP can effectively reconstruct training images according to the gradients of soft prompts or adapters of CLIP models

Protect Federated Learning Against Backdoor Attacks via Data-Free Trigger Generation

As a distributed machine learning paradigm, Federated Learning (FL) enables large-scale clients to collaboratively train a model without sharing their raw data. However, due to the lack of data auditing for untrusted clients, FL is vulnerable to poisoning attacks, especially backdoor attacks. By using poisoned data for local training or directly changing the model parameters, attackers can easily inject backdoors into the model, which can trigger the model to make misclassification of targeted patterns in images. To address these issues, we propose a novel data-free trigger-generation-based defense approach based on the two characteristics of backdoor attacks: i) triggers are learned faster than normal knowledge, and ii) trigger patterns have a greater effect on image classification than normal class patterns. Our approach generates the images with newly learned knowledge by identifying the differences between the old and new global models, and filters trigger images by evaluating the effect of these generated images. By using these trigger images, our approach eliminates poisoned models to ensure the updated global model is benign. Comprehensive experiments demonstrate that our approach can defend against almost all the existing types of backdoor attacks and outperform all the seven state-of-the-art defense methods with both IID and non-IID scenarios. Especially, our approach can successfully defend against the backdoor attack even when 80\% of the clients are malicious

GitFL: Adaptive Asynchronous Federated Learning using Version Control

As a promising distributed machine learning paradigm that enables collaborative training without compromising data privacy, Federated Learning (FL) has been increasingly used in AIoT (Artificial Intelligence of Things) design. However, due to the lack of efficient management of straggling devices, existing FL methods greatly suffer from the problems of low inference accuracy and long training time. Things become even worse when taking various uncertain factors (e.g., network delays, performance variances caused by process variation) existing in AIoT scenarios into account. To address this issue, this paper proposes a novel asynchronous FL framework named GitFL, whose implementation is inspired by the famous version control system Git. Unlike traditional FL, the cloud server of GitFL maintains a master model (i.e., the global model) together with a set of branch models indicating the trained local models committed by selected devices, where the master model is updated based on both all the pushed branch models and their version information, and only the branch models after the pull operation are dispatched to devices. By using our proposed Reinforcement Learning (RL)-based device selection mechanism, a pulled branch model with an older version will be more likely to be dispatched to a faster and less frequently selected device for the next round of local training. In this way, GitFL enables both effective control of model staleness and adaptive load balance of versioned models among straggling devices, thus avoiding the performance deterioration. Comprehensive experimental results on well-known models and datasets show that, compared with state-of-the-art asynchronous FL methods, GitFL can achieve up to 2.64X training acceleration and 7.88% inference accuracy improvements in various uncertain scenarios

Building a digital twin of EDFA: a grey-box modeling approach

To enable intelligent and self-driving optical networks, high-accuracy physical layer models are required. The dynamic wavelength-dependent gain effects of non-constant-pump erbium-doped fiber amplifiers (EDFAs) remain a crucial problem in terms of modeling, as it determines optical-to-signal noise ratio as well as the magnitude of fiber nonlinearities. Black-box data-driven models have been widely studied, but it requires a large size of data for training and suffers from poor generalizability. In this paper, we derive the gain spectra of EDFAs as a simple univariable linear function, and then based on it we propose a grey-box EDFA gain modeling scheme. Experimental results show that for both automatic gain control (AGC) and automatic power control (APC) EDFAs, our model built with 8 data samples can achieve better performance than the neural network (NN) based model built with 900 data samples, which means the required data size for modeling can be reduced by at least two orders of magnitude. Moreover, in the experiment the proposed model demonstrates superior generalizability to unseen scenarios since it is based on the underlying physics of EDFAs. The results indicate that building a customized digital twin of each EDFA in optical networks become feasible, which is essential especially for next generation multi-band network operations

Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models

Although recent personalization methods have democratized high-resolution image synthesis by enabling swift concept acquisition with minimal examples and lightweight computation, they also present an exploitable avenue for high accessible backdoor attacks. This paper investigates a critical and unexplored aspect of text-to-image (T2I) diffusion models - their potential vulnerability to backdoor attacks via personalization. Our study focuses on a zero-day backdoor vulnerability prevalent in two families of personalization methods, epitomized by Textual Inversion and DreamBooth.Compared to traditional backdoor attacks, our proposed method can facilitate more precise, efficient, and easily accessible attacks with a lower barrier to entry. We provide a comprehensive review of personalization in T2I diffusion models, highlighting the operation and exploitation potential of this backdoor vulnerability. To be specific, by studying the prompt processing of Textual Inversion and DreamBooth, we have devised dedicated backdoor attacks according to the different ways of dealing with unseen tokens and analyzed the influence of triggers and concept images on the attack effect. Through comprehensive empirical study, we endorse the utilization of the nouveau-token backdoor attack due to its impressive effectiveness, stealthiness, and integrity, markedly outperforming the legacy-token backdoor attack.Comment: 16 pages, accepted by AAAI 202

Moderating effect of classroom sociable norm on the relations between unsociability and internalizing problems in Chinese adolescents

ObjectivesThe goal of the present study was to examine the moderating effect of classroom sociable norm on the relations between unsociability and internalizing problems (the indicators included depression, loneliness and self-esteem) in Chinese adolescents.MethodsParticipants were N = 1,160 adolescents in Grade 4–8 from Shanghai, People’s Republic of China. They completed questionnaires about unsociability, sociability, and social preference via peer nominations, while depression, loneliness, and self-esteem were collected via self-report.ResultsIt was found that unsociability was positively associated with depression and loneliness, and negatively associated with self-esteem. Moreover, the relations between unsociability and indicators of internalizing problems were moderated by classroom sociable norm. More specifically, the significant positive associations between unsociability and depression and loneliness were stronger in classrooms with high sociable norm, and the negative association between unsociability and self-esteem was only significant in such classrooms.ConclusionThe findings suggest that classroom sociable norm plays an important role in unsociable adolescents’ psychological adjustment in China. Researchers should focus more on the influence of classroom environment on adolescents’ development in future

Observation of photonic antichiral edge states

Chiral edge states are a hallmark feature of two-dimensional topological materials. Such states must propagate along the edges of the bulk either clockwise or counterclockwise, and thus produce oppositely propagating edge states along the two parallel edges of a strip sample. However, recent theories have predicted a counterintuitive picture, where the two edge states at the two parallel strip edges can propagate in the same direction; these anomalous topological edge states are named as antichiral edge states. Here we report the experimental observation of antichiral edge states in a gyromagnetic photonic crystal. The crystal consists of gyromagnetic cylinders in a honeycomb lattice, with the two triangular sublattices magnetically biased in opposite directions. With microwave measurement, unique properties of antichiral edge states have been observed directly, which include the titled dispersion, the chiral-like robust propagation in samples with certain shapes, and the scattering into backward bulk states at certain terminations. These results extend and supplement the current understanding of chiral edge states

L-arginine combination with 5-fluorouracil inhibit hepatocellular carcinoma cells through suppressing iNOS/NO/AKT-mediated glycolysis

L-arginine can produce nitric oxide (NO) under the action of inducible nitric oxide synthase (iNOS), while 5-fluorouracil (5-FU) can induce the increase of iNOS expression. The present study was to investigate the mechanism of L-arginine combined with 5-FU regulating glucose metabolism of hepatocellular carcinoma (HCC) through iNOS/NO/AKT pathway. The combination of L-arginine and 5-FU resulted in decreased cell survival and exhibited synergistic cytotoxic effects in HepG2 and SMMC7721 cells. Meanwhile, L-arginine increased 5-FU inhibitory effect on HepG2 and SMMC7721 cells by increasing NO production. Co-treatment with L-arginine and 5-FU resulted in a significant decrease in both G6PDH and LDH enzymatic activities, as well as reduced levels of ATP and LD compared to treatment with L-arginine or 5-FU alone. Moreover, the combination of L-arginine and 5-FU resulted in a decrease in the expression of GLUT1, PKM2, LDHA, p-PI3K and p-AKT. Furthermore, the combination demonstrated a synergistic effect in downregulating the expression of HIF-1α and β-catenin, which were further diminished upon the addition of shikonin, a specific inhibitor of PKM2. LY294002 treatment further reduced the expression of GLUT1, PKM2, and LDHA proteins induced by combined L-arginine and 5-FU treatment compared to the combined group. However, the reduction in p-PI3K, p-AKT, and GLUT1 expression caused by L-arginine and 5-FU combination was also reversed in HepG2 and SMMC7721 cells with iNOS knockdown, respectively. Additionally, the combination of L-arginine and 5-FU led to a greater reduction in the enzymatic activity of ALT, AST, G6PDH and LDH, as well as a significant reduction in hepatic index, AFP, AFP-L3, ATP and LD levels in a rat model of HCC. Moreover, the simultaneous administration of L-arginine and 5-FU significantly improved the gross morphology of the liver, reduced nuclear atypia, inhibited the proliferation of cancer cells, and decreased the expression levels of p-PI3K, p-AKT, GLUT1, PKM2, and LDHA, while iNOS expression was increased in the combination group. Taking together, L-arginine and 5-FU combination resulted in the inhibition of enzymes in aerobic glycolysis via the iNOS/NO/AKT pathway, which led to the suppression of glucose metabolism and downregulation of nuclear transcription factors, thereby impeding the proliferation of hepatocellular carcinoma cells