19 research outputs found
Ske2Grid: Skeleton-to-Grid Representation Learning for Action Recognition
This paper presents Ske2Grid, a new representation learning framework for
improved skeleton-based action recognition. In Ske2Grid, we define a regular
convolution operation upon a novel grid representation of human skeleton, which
is a compact image-like grid patch constructed and learned through three novel
designs. Specifically, we propose a graph-node index transform (GIT) to
construct a regular grid patch through assigning the nodes in the skeleton
graph one by one to the desired grid cells. To ensure that GIT is a bijection
and enrich the expressiveness of the grid representation, an up-sampling
transform (UPT) is learned to interpolate the skeleton graph nodes for filling
the grid patch to the full. To resolve the problem when the one-step UPT is
aggressive and further exploit the representation capability of the grid patch
with increasing spatial size, a progressive learning strategy (PLS) is proposed
which decouples the UPT into multiple steps and aligns them to multiple paired
GITs through a compact cascaded design learned progressively. We construct
networks upon prevailing graph convolution networks and conduct experiments on
six mainstream skeleton-based action recognition datasets. Experiments show
that our Ske2Grid significantly outperforms existing GCN-based solutions under
different benchmark settings, without bells and whistles. Code and models are
available at https://github.com/OSVAI/Ske2GridComment: The paper of Ske2Grid is published at ICML 2023. Code and models are
available at https://github.com/OSVAI/Ske2Gri
FwdLLM: Efficient FedLLM using Forward Gradient
Large Language Models (LLMs) are transforming the landscape of mobile
intelligence. Federated Learning (FL), a method to preserve user data privacy,
is often employed in fine-tuning LLMs to downstream mobile tasks, an approach
known as FedLLM. Though recent efforts have addressed the network issue induced
by the vast model size, they have not practically mitigated vital challenges
concerning integration with mobile devices, such as significant memory
consumption and sluggish model convergence.
In response to these challenges, this work introduces FwdLLM, an innovative
FL protocol designed to enhance the FedLLM efficiency. The key idea of FwdLLM
to employ backpropagation (BP)-free training methods, requiring devices only to
execute ``perturbed inferences''. Consequently, FwdLLM delivers way better
memory efficiency and time efficiency (expedited by mobile NPUs and an expanded
array of participant devices). FwdLLM centers around three key designs: (1) it
combines BP-free training with parameter-efficient training methods, an
essential way to scale the approach to the LLM era; (2) it systematically and
adaptively allocates computational loads across devices, striking a careful
balance between convergence speed and accuracy; (3) it discriminatively samples
perturbed predictions that are more valuable to model convergence.
Comprehensive experiments with five LLMs and three NLP tasks illustrate
FwdLLM's significant advantages over conventional methods, including up to
three orders of magnitude faster convergence and a 14.6x reduction in memory
footprint. Uniquely, FwdLLM paves the way for federated learning of
billion-parameter LLMs such as LLaMA on COTS mobile devices -- a feat
previously unattained.Comment: under revie
Federated NLP in Few-shot Scenarios
Natural language processing (NLP) sees rich mobile applications. To support
various language understanding tasks, a foundation NLP model is often
fine-tuned in a federated, privacy-preserving setting (FL). This process
currently relies on at least hundreds of thousands of labeled training samples
from mobile clients; yet mobile users often lack willingness or knowledge to
label their data. Such an inadequacy of data labels is known as a few-shot
scenario; it becomes the key blocker for mobile NLP applications.
For the first time, this work investigates federated NLP in the few-shot
scenario (FedFSL). By retrofitting algorithmic advances of pseudo labeling and
prompt learning, we first establish a training pipeline that delivers
competitive accuracy when only 0.05% (fewer than 100) of the training data is
labeled and the remaining is unlabeled. To instantiate the workflow, we further
present a system FFNLP, addressing the high execution cost with novel designs.
(1) Curriculum pacing, which injects pseudo labels to the training workflow at
a rate commensurate to the learning progress; (2) Representational diversity, a
mechanism for selecting the most learnable data, only for which pseudo labels
will be generated; (3) Co-planning of a model's training depth and layer
capacity. Together, these designs reduce the training delay, client energy, and
network traffic by up to 46.0, 41.2 and 3000.0,
respectively. Through algorithm/system co-design, FFNLP demonstrates that FL
can apply to challenging settings where most training samples are unlabeled
Towards Practical Few-shot Federated NLP
Transformer-based pre-trained models have emerged as the predominant solution
for natural language processing (NLP). Fine-tuning such pre-trained models for
downstream tasks often requires a considerable amount of labeled private data.
In practice, private data is often distributed across heterogeneous mobile
devices and may be prohibited from being uploaded. Moreover, well-curated
labeled data is often scarce, presenting an additional challenge. To address
these challenges, we first introduce a data generator for federated few-shot
learning tasks, which encompasses the quantity and skewness of scarce labeled
data in a realistic setting. Subsequently, we propose AUG-FedPrompt, a
prompt-based federated learning system that exploits abundant unlabeled data
for data augmentation. Our experiments indicate that AUG-FedPrompt can perform
on par with full-set fine-tuning with a limited amount of labeled data.
However, such competitive performance comes at a significant system cost.Comment: EuroSys23 worksho
Accelerating Vertical Federated Learning
Privacy, security and data governance constraints rule out a brute force
process in the integration of cross-silo data, which inherits the development
of the Internet of Things. Federated learning is proposed to ensure that all
parties can collaboratively complete the training task while the data is not
out of the local. Vertical federated learning is a specialization of federated
learning for distributed features. To preserve privacy, homomorphic encryption
is applied to enable encrypted operations without decryption. Nevertheless,
together with a robust security guarantee, homomorphic encryption brings extra
communication and computation overhead. In this paper, we analyze the current
bottlenecks of vertical federated learning under homomorphic encryption
comprehensively and numerically. We propose a straggler-resilient and
computation-efficient accelerating system that reduces the communication
overhead in heterogeneous scenarios by 65.26% at most and reduces the
computation overhead caused by homomorphic encryption by 40.66% at most. Our
system can improve the robustness and efficiency of the current vertical
federated learning framework without loss of security
Mitofusin 2 Participates in Mitophagy and Mitochondrial Fusion Against Angiotensin II-Induced Cardiomyocyte Injury
BackgroundMitochondrial dynamics play a critical role in mitochondrial function. The mitofusin 2 (MFN2) gene encodes a mitochondrial membrane protein that participates in mitochondrial fusion to maintain and operate the mitochondrial network. Moreover, MFN2 is essential for mitophagy. In Ang II-induced cardiac remodeling, the combined effects of MFN2-mediated mitochondrial fusion and mitophagy are unclear. This study was designed to explore a novel strategy for preventing cardiomyocyte injury via modulation of mitochondrial dynamics.MethodsWe studied the function of MFN2 in mitochondrial fusion and mitophagy in Ang II-stimulated cardiomyocyte injury. Cardiomyocyte injury experiments, including reactive oxygen species (ROS) production, mitochondrial membrane potential (MMP), and apoptosis rate of cardiomyocytes were performed. The mitochondrial morphology in cardiomyocytes was examined via transmission electron microscopy (TEM) and confocal microscopy. Autophagic levels in response to Ang II were examined by immunoblotting of autophagy-related proteins. Moreover, PINK1/MFN2/Parkin pathway-related proteins were examined.ResultsWith stimulation by Ang II, MFN2 expression was progressively reduced. MFN2 deficiency impaired mitochondrial quality, resulting in exacerbated mitochondrial damage induced by Ang II. The Ang II-induced increases in ROS production and apoptosis rate were alleviated by MFN2 overexpression. Moreover, MFN2 alleviated the Ang II-induced reduction in MMP. MFN2 promoted mitochondrial fusion, and MFN2 promoted Parkin translocation and phosphorylation, leading to mitochondrial autophagy. The effects of MFN2 overexpression were reversed by autophagy inhibitors.ConclusionMitofusin 2 promotes Parkin translocation and phosphorylation, leading to mitophagy to clear damaged mitochondria. However, the beneficial effects of MFN2 were reversed by autophagy inhibitors. Additionally, MFN2 participates in mitochondrial fusion to maintain mitochondrial quality. Thus, MFN2 participated in mitophagy and mitochondrial fusion against Ang II-induced cardiomyocyte injury
Lightweight Protection for Privacy in Offloaded Speech Understanding
Speech is a common input method for mobile embedded devices, but cloud-based
speech recognition systems pose privacy risks. Disentanglement-based encoders,
designed to safeguard user privacy by filtering sensitive information from
speech signals, unfortunately require substantial memory and computational
resources, which limits their use in less powerful devices. To overcome this,
we introduce a novel system, XXX, optimized for such devices. XXX is built on
the insight that speech understanding primarily relies on understanding the
entire utterance's long-term dependencies, while privacy concerns are often
linked to short-term details. Therefore, XXX focuses on selectively masking
these short-term elements, preserving the quality of long-term speech
understanding. The core of XXX is an innovative differential mask generator,
grounded in interpretable learning, which fine-tunes the masking process. We
tested XXX on the STM32H7 microcontroller, assessing its performance in various
potential attack scenarios. The results show that XXX maintains speech
understanding accuracy and privacy at levels comparable to existing encoders,
but with a significant improvement in efficiency, achieving up to 53.3
faster processing and a 134.1 smaller memory footprint.Comment: arXiv admin comment: This version has been removed by arXiv
administrators as the submitter did not have the rights to agree to the
license at the time of submissio
An Electrochromic Ag-Decorated WO<sub>3−x</sub> Film with Adjustable Defect States for Electrochemical Surface-Enhanced Raman Spectroscopy
Electrochemical surface-enhanced Raman scattering (EC-SERS) spectroscopy is an ultrasensitive spectro-electrochemistry technique that provides mechanistic and dynamic information on electrochemical interfaces at the molecular level. However, the plasmon-mediated photocatalysis hinders the intrinsic electrochemical behavior of molecules at electrochemical interfaces. This work aimed to develop a facile method for constructing a reliable EC-SERS substrate that can be used to study the molecular dynamics at electrochemical interfaces. Herein, a novel Ag-WO3−x electrochromic heterostructure was synthesized for EC-SERS. Especially, the use of electrochromic WO3−x film suppresses the influence of hot-electrons-induced catalysis while offering a reliable SERS effect. Based on this finding, the real electrochemical behavior of p-aminothiophenol (PATP) on Ag nanoparticles (NPs) surface was revealed for the first time. We are confident that metal-semiconductor electrochromic heterostructures could be developed into reliable substrates for EC-SERS analysis. Furthermore, the results obtained in this work provide new insights not only into the chemical mechanism of SERS, but also into the hot-electron transfer mechanism in metal-semiconductor heterostructures