9,751 research outputs found
Decentralized Collaborative Learning of Personalized Models over Networks
We consider a set of learning agents in a col-laborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective. The question addressed in this paper is: how can agents improve upon their locally trained model by communicating with other agents that have similar objectives? We introduce and analyze two asynchronous gossip algorithms running in a fully decentralized manner. Our first approach , inspired from label propagation, aims to smooth pre-trained local models over the network while accounting for the confidence that each agent has in its initial model. In our second approach, agents jointly learn and propagate their model by making iterative updates based on both their local dataset and the behavior of their neighbors. Our algorithm to optimize this challenging objective in a decentralized way is based on ADMM
Decentralized Collaborative Learning Framework for Next POI Recommendation
Next Point-of-Interest (POI) recommendation has become an indispensable
functionality in Location-based Social Networks (LBSNs) due to its
effectiveness in helping people decide the next POI to visit. However, accurate
recommendation requires a vast amount of historical check-in data, thus
threatening user privacy as the location-sensitive data needs to be handled by
cloud servers. Although there have been several on-device frameworks for
privacy-preserving POI recommendations, they are still resource-intensive when
it comes to storage and computation, and show limited robustness to the high
sparsity of user-POI interactions. On this basis, we propose a novel
decentralized collaborative learning framework for POI recommendation (DCLR),
which allows users to train their personalized models locally in a
collaborative manner. DCLR significantly reduces the local models' dependence
on the cloud for training, and can be used to expand arbitrary centralized
recommendation models. To counteract the sparsity of on-device user data when
learning each local model, we design two self-supervision signals to pretrain
the POI representations on the server with geographical and categorical
correlations of POIs. To facilitate collaborative learning, we innovatively
propose to incorporate knowledge from either geographically or semantically
similar users into each local model with attentive aggregation and mutual
information maximization. The collaborative learning process makes use of
communications between devices while requiring only minor engagement from the
central server for identifying user groups, and is compatible with common
privacy preservation mechanisms like differential privacy. We evaluate DCLR
with two real-world datasets, where the results show that DCLR outperforms
state-of-the-art on-device frameworks and yields competitive results compared
with centralized counterparts.Comment: 21 Pages, 3 figures, 4 table
Fog-enabled Edge Learning for Cognitive Content-Centric Networking in 5G
By caching content at network edges close to the users, the content-centric
networking (CCN) has been considered to enforce efficient content retrieval and
distribution in the fifth generation (5G) networks. Due to the volume,
velocity, and variety of data generated by various 5G users, an urgent and
strategic issue is how to elevate the cognitive ability of the CCN to realize
context-awareness, timely response, and traffic offloading for 5G applications.
In this article, we envision that the fundamental work of designing a cognitive
CCN (C-CCN) for the upcoming 5G is exploiting the fog computing to
associatively learn and control the states of edge devices (such as phones,
vehicles, and base stations) and in-network resources (computing, networking,
and caching). Moreover, we propose a fog-enabled edge learning (FEL) framework
for C-CCN in 5G, which can aggregate the idle computing resources of the
neighbouring edge devices into virtual fogs to afford the heavy delay-sensitive
learning tasks. By leveraging artificial intelligence (AI) to jointly
processing sensed environmental data, dealing with the massive content
statistics, and enforcing the mobility control at network edges, the FEL makes
it possible for mobile users to cognitively share their data over the C-CCN in
5G. To validate the feasibility of proposed framework, we design two
FEL-advanced cognitive services for C-CCN in 5G: 1) personalized network
acceleration, 2) enhanced mobility management. Simultaneously, we present the
simulations to show the FEL's efficiency on serving for the mobile users'
delay-sensitive content retrieval and distribution in 5G.Comment: Submitted to IEEE Communications Magzine, under review, Feb. 09, 201
Model-Agnostic Decentralized Collaborative Learning for On-Device POI Recommendation
As an indispensable personalized service in Location-based Social Networks
(LBSNs), the next Point-of-Interest (POI) recommendation aims to help people
discover attractive and interesting places. Currently, most POI recommenders
are based on the conventional centralized paradigm that heavily relies on the
cloud to train the recommendation models with large volumes of collected users'
sensitive check-in data. Although a few recent works have explored on-device
frameworks for resilient and privacy-preserving POI recommendations, they
invariably hold the assumption of model homogeneity for parameters/gradients
aggregation and collaboration. However, users' mobile devices in the real world
have various hardware configurations (e.g., compute resources), leading to
heterogeneous on-device models with different architectures and sizes. In light
of this, We propose a novel on-device POI recommendation framework, namely
Model-Agnostic Collaborative learning for on-device POI recommendation (MAC),
allowing users to customize their own model structures (e.g., dimension \&
number of hidden layers). To counteract the sparsity of on-device user data, we
propose to pre-select neighbors for collaboration based on physical distances,
category-level preferences, and social networks. To assimilate knowledge from
the above-selected neighbors in an efficient and secure way, we adopt the
knowledge distillation framework with mutual information maximization. Instead
of sharing sensitive models/gradients, clients in MAC only share their soft
decisions on a preloaded reference dataset. To filter out low-quality
neighbors, we propose two sampling strategies, performance-triggered sampling
and similarity-based sampling, to speed up the training process and obtain
optimal recommenders. In addition, we design two novel approaches to generate
more effective reference datasets while protecting users' privacy
Heterogeneous Collaborative Learning for Personalized Healthcare Analytics via Messenger Distillation
In this paper, we propose a Similarity-Quality-based Messenger Distillation
(SQMD) framework for heterogeneous asynchronous on-device healthcare analytics.
By introducing a preloaded reference dataset, SQMD enables all participant
devices to distill knowledge from peers via messengers (i.e., the soft labels
of the reference dataset generated by clients) without assuming the same model
architecture. Furthermore, the messengers also carry important auxiliary
information to calculate the similarity between clients and evaluate the
quality of each client model, based on which the central server creates and
maintains a dynamic collaboration graph (communication graph) to improve the
personalization and reliability of SQMD under asynchronous conditions.
Extensive experiments on three real-life datasets show that SQMD achieves
superior performance
- …