497 research outputs found
Edge Intelligence : Empowering Intelligence to the Edge of Network
Edge intelligence refers to a set of connected systems and devices for data collection, caching, processing, and analysis proximity to where data are captured based on artificial intelligence. Edge intelligence aims at enhancing data processing and protects the privacy and security of the data and users. Although recently emerged, spanning the period from 2011 to now, this field of research has shown explosive growth over the past five years. In this article, we present a thorough and comprehensive survey of the literature surrounding edge intelligence. We first identify four fundamental components of edge intelligence, i.e., edge caching, edge training, edge inference, and edge offloading based on theoretical and practical results pertaining to proposed and deployed systems. We then aim for a systematic classification of the state of the solutions by examining research results and observations for each of the four components and present a taxonomy that includes practical problems, adopted techniques, and application goals. For each category, we elaborate, compare, and analyze the literature from the perspectives of adopted techniques, objectives, performance, advantages and drawbacks, and so on. This article provides a comprehensive survey of edge intelligence and its application areas. In addition, we summarize the development of the emerging research fields and the current state of the art and discuss the important open issues and possible theoretical and technical directions.Peer reviewe
FLuID: Mitigating Stragglers in Federated Learning using Invariant Dropout
Federated Learning (FL) allows machine learning models to train locally on
individual mobile devices, synchronizing model updates via a shared server.
This approach safeguards user privacy; however, it also generates a
heterogeneous training environment due to the varying performance capabilities
across devices. As a result, straggler devices with lower performance often
dictate the overall training time in FL. In this work, we aim to alleviate this
performance bottleneck due to stragglers by dynamically balancing the training
load across the system. We introduce Invariant Dropout, a method that extracts
a sub-model based on the weight update threshold, thereby minimizing potential
impacts on accuracy. Building on this dropout technique, we develop an adaptive
training framework, Federated Learning using Invariant Dropout (FLuID). FLuID
offers a lightweight sub-model extraction to regulate computational intensity,
thereby reducing the load on straggler devices without affecting model quality.
Our method leverages neuron updates from non-straggler devices to construct a
tailored sub-model for each straggler based on client performance profiling.
Furthermore, FLuID can dynamically adapt to changes in stragglers as runtime
conditions shift. We evaluate FLuID using five real-world mobile clients. The
evaluations show that Invariant Dropout maintains baseline model efficiency
while alleviating the performance bottleneck of stragglers through a dynamic,
runtime approach
Efficient Federated Learning with Enhanced Privacy via Lottery Ticket Pruning in Edge Computing
Federated learning (FL) is a collaborative learning paradigm for
decentralized private data from mobile terminals (MTs). However, it suffers
from issues in terms of communication, resource of MTs, and privacy. Existing
privacy-preserving FL methods usually adopt the instance-level differential
privacy (DP), which provides a rigorous privacy guarantee but with several
bottlenecks: severe performance degradation, transmission overhead, and
resource constraints of edge devices such as MTs. To overcome these drawbacks,
we propose Fed-LTP, an efficient and privacy-enhanced FL framework with
\underline{\textbf{L}}ottery \underline{\textbf{T}}icket
\underline{\textbf{H}}ypothesis (LTH) and zero-concentrated
D\underline{\textbf{P}} (zCDP). It generates a pruned global model on the
server side and conducts sparse-to-sparse training from scratch with zCDP on
the client side. On the server side, two pruning schemes are proposed: (i) the
weight-based pruning (LTH) determines the pruned global model structure; (ii)
the iterative pruning further shrinks the size of the pruned model's
parameters. Meanwhile, the performance of Fed-LTP is also boosted via model
validation based on the Laplace mechanism. On the client side, we use
sparse-to-sparse training to solve the resource-constraints issue and provide
tighter privacy analysis to reduce the privacy budget. We evaluate the
effectiveness of Fed-LTP on several real-world datasets in both independent and
identically distributed (IID) and non-IID settings. The results clearly confirm
the superiority of Fed-LTP over state-of-the-art (SOTA) methods in
communication, computation, and memory efficiencies while realizing a better
utility-privacy trade-off.Comment: 13 page
- …