4,551 research outputs found
Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
On-device machine learning (ML) enables the training process to exploit a
massive amount of user-generated private data samples. To enjoy this benefit,
inter-device communication overhead should be minimized. With this end, we
propose federated distillation (FD), a distributed model training algorithm
whose communication payload size is much smaller than a benchmark scheme,
federated learning (FL), particularly when the model size is large. Moreover,
user-generated data samples are likely to become non-IID across devices, which
commonly degrades the performance compared to the case with an IID dataset. To
cope with this, we propose federated augmentation (FAug), where each device
collectively trains a generative model, and thereby augments its local data
towards yielding an IID dataset. Empirical studies demonstrate that FD with
FAug yields around 26x less communication overhead while achieving 95-98% test
accuracy compared to FL.Comment: presented at the 32nd Conference on Neural Information Processing
Systems (NIPS 2018), 2nd Workshop on Machine Learning on the Phone and other
Consumer Devices (MLPCD 2), Montr\'eal, Canad
DeepStory: Video Story QA by Deep Embedded Memory Networks
Question-answering (QA) on video contents is a significant challenge for
achieving human-level intelligence as it involves both vision and language in
real-world settings. Here we demonstrate the possibility of an AI agent
performing video story QA by learning from a large amount of cartoon videos. We
develop a video-story learning model, i.e. Deep Embedded Memory Networks
(DEMN), to reconstruct stories from a joint scene-dialogue video stream using a
latent embedding space of observed data. The video stories are stored in a
long-term memory component. For a given question, an LSTM-based attention model
uses the long-term memory to recall the best question-story-answer triplet by
focusing on specific words containing key information. We trained the DEMN on a
novel QA dataset of children's cartoon video series, Pororo. The dataset
contains 16,066 scene-dialogue pairs of 20.5-hour videos, 27,328 fine-grained
sentences for scene description, and 8,913 story-related QA pairs. Our
experimental results show that the DEMN outperforms other QA models. This is
mainly due to 1) the reconstruction of video stories in a scene-dialogue
combined form that utilize the latent embedding and 2) attention. DEMN also
achieved state-of-the-art results on the MovieQA benchmark.Comment: 7 pages, accepted for IJCAI 201
Epidemic Response Coordination Networks in âLiving Documentsâ
Response plans developed thoroughly are suggestive of a successful action, but there is a gap in the literature with respect to the way concerted efforts among organizations are planned and change during crises. Using organizational network data extracted from the South Korean governmentâs MERS response manuals, we examined the changes in the response coordination network planned during the epidemicâs distinct stages. The greatest difference in predicting tie formation was found in the networks planned before the event and revised during the outbreak. Local and governmental actors tend to form more ties consistently in the revised manuals. Two actors that are intended to transfer medical and/or personnel resources tend to form more ties across all stages. These findings suggest that transferring material and/or human resources are key activities in the epidemic response and planners tend to increase the connection of local and governmental actors over time
SplitAMC: Split Learning for Robust Automatic Modulation Classification
Automatic modulation classification (AMC) is a technology that identifies a
modulation scheme without prior signal information and plays a vital role in
various applications, including cognitive radio and link adaptation. With the
development of deep learning (DL), DL-based AMC methods have emerged, while
most of them focus on reducing computational complexity in a centralized
structure. This centralized learning-based AMC (CentAMC) violates data privacy
in the aspect of direct transmission of client-side raw data. Federated
learning-based AMC (FedeAMC) can bypass this issue by exchanging model
parameters, but causes large resultant latency and client-side computational
load. Moreover, both CentAMC and FedeAMC are vulnerable to large-scale noise
occured in the wireless channel between the client and the server. To this end,
we develop a novel AMC method based on a split learning (SL) framework, coined
SplitAMC, that can achieve high accuracy even in poor channel conditions, while
guaranteeing data privacy and low latency. In SplitAMC, each client can benefit
from data privacy leakage by exchanging smashed data and its gradient instead
of raw data, and has robustness to noise with the help of high scale of smashed
data. Numerical evaluations validate that SplitAMC outperforms CentAMC and
FedeAMC in terms of accuracy for all SNRs as well as latency.Comment: to be presented at IEEE VTC2023-Sprin
Improved Chest Anomaly Localization without Pixel-level Annotation via Image Translation Network Application in Pseudo-paired Registration Domain
Image translation based on a generative adversarial network (GAN-IT) is a
promising method for the precise localization of abnormal regions in chest
X-ray images (AL-CXR) even without pixel-level annotation. However,
heterogeneous unpaired datasets undermine existing methods to extract key
features and distinguish normal from abnormal cases, resulting in inaccurate
and unstable AL-CXR. To address this problem, we propose an improved two-stage
GAN-IT involving registration and data augmentation. For the first stage, we
introduce an advanced deep-learning-based registration technique that virtually
and reasonably converts unpaired data into paired data for learning
registration maps, by sequentially utilizing linear-based global and uniform
coordinate transformation and AI-based non-linear coordinate fine-tuning. This
approach enables the independent and complex coordinate transformation of each
detailed location of the lung while recognizing the entire lung structure,
thereby achieving higher registration performance with resolving inherent
artifacts caused by unpaired conditions. For the second stage, we apply data
augmentation to diversify anomaly locations by swapping the left and right lung
regions on the uniform registered frames, further improving the performance by
alleviating imbalance in data distribution showing left and right lung lesions.
The proposed method is model agnostic and shows consistent AL-CXR performance
improvement in representative AI models. Therefore, we believe GAN-IT for
AL-CXR can be clinically implemented by using our basis framework, even if
learning data are scarce or difficult for the pixel-level disease annotation
- âŠ