1,159 research outputs found
Task-Oriented Communication for Multi-Device Cooperative Edge Inference
This paper investigates task-oriented communication for multi-device
cooperative edge inference, where a group of distributed low-end edge devices
transmit the extracted features of local samples to a powerful edge server for
inference. While cooperative edge inference can overcome the limited sensing
capability of a single device, it substantially increases the communication
overhead and may incur excessive latency. To enable low-latency cooperative
inference, we propose a learning-based communication scheme that optimizes
local feature extraction and distributed feature encoding in a task-oriented
manner, i.e., to remove data redundancy and transmit information that is
essential for the downstream inference task rather than reconstructing the data
samples at the edge server. Specifically, we leverage an information bottleneck
(IB) principle to extract the task-relevant feature at each edge device and
adopt a distributed information bottleneck (DIB) framework to formalize a
single-letter characterization of the optimal rate-relevance tradeoff for
distributed feature encoding. To admit flexible control of the communication
overhead, we extend the DIB framework to a distributed deterministic
information bottleneck (DDIB) objective that explicitly incorporates the
representational costs of the encoded features. As the IB-based objectives are
computationally prohibitive for high-dimensional data, we adopt variational
approximations to make the optimization problems tractable. To compensate the
potential performance loss due to the variational approximations, we also
develop a selective retransmission (SR) mechanism to identify the redundancy in
the encoded features of multiple edge devices to attain additional
communication overhead reduction. Extensive experiments evidence that the
proposed task-oriented communication scheme achieves a better rate-relevance
tradeoff than baseline methods.Comment: This paper was accepted to IEEE Transactions on Wireless
Communicatio
Enhanced superconductivity at the interface of W/SrRuO point contact
Differential resistance measurements are conducted for point contacts (PCs)
between tungsten tip approaching along the axis direction and the
plane of SrRuO single crystal. Three key features are found.
Firstly, within 0.2 mV there is a dome like conductance enhancement due to
Andreev reflection at the normal-superconducting interface. By pushing the W
tip further, the conductance enhancement increases from 3\% to more than 20\%,
much larger than that was previously reported, probably due to the pressure
exerted by the tip. Secondly, there are also superconducting like features at
bias higher than 0.2 mV which persists up to 6.2 K, resembling the enhanced
superconductivity under uniaxial pressure for bulk SrRuO crystals
but more pronounced here. Third, the logarithmic background can be fitted with
the Altshuler-Aronov theory of tunneling into quasi two dimensional electron
system, consistent with the highly anisotropic electronic system in
SrRuO.Comment: prb style, 9 pages, 8 fig
Branchy-GNN: a Device-Edge Co-Inference Framework for Efficient Point Cloud Processing
The recent advancements of three-dimensional (3D) data acquisition devices
have spurred a new breed of applications that rely on point cloud data
processing. However, processing a large volume of point cloud data brings a
significant workload on resource-constrained mobile devices, prohibiting from
unleashing their full potentials. Built upon the emerging paradigm of
device-edge co-inference, where an edge device extracts and transmits the
intermediate feature to an edge server for further processing, we propose
Branchy-GNN for efficient graph neural network (GNN) based point cloud
processing by leveraging edge computing platforms. In order to reduce the
on-device computational cost, the Branchy-GNN adds branch networks for early
exiting. Besides, it employs learning-based joint source-channel coding (JSCC)
for the intermediate feature compression to reduce the communication overhead.
Our experimental results demonstrate that the proposed Branchy-GNN secures a
significant latency reduction compared with several benchmark methods
Hierarchical Text Classification with Reinforced Label Assignment
While existing hierarchical text classification (HTC) methods attempt to
capture label hierarchies for model training, they either make local decisions
regarding each label or completely ignore the hierarchy information during
inference. To solve the mismatch between training and inference as well as
modeling label dependencies in a more principled way, we formulate HTC as a
Markov decision process and propose to learn a Label Assignment Policy via deep
reinforcement learning to determine where to place an object and when to stop
the assignment process. The proposed method, HiLAP, explores the hierarchy
during both training and inference time in a consistent manner and makes
inter-dependent decisions. As a general framework, HiLAP can incorporate
different neural encoders as base models for end-to-end training. Experiments
on five public datasets and four base models show that HiLAP yields an average
improvement of 33.4% in Macro-F1 over flat classifiers and outperforms
state-of-the-art HTC methods by a large margin. Data and code can be found at
https://github.com/morningmoni/HiLAP.Comment: EMNLP 201
End-to-End Reinforcement Learning for Automatic Taxonomy Induction
We present a novel end-to-end reinforcement learning approach to automatic
taxonomy induction from a set of terms. While prior methods treat the problem
as a two-phase task (i.e., detecting hypernymy pairs followed by organizing
these pairs into a tree-structured hierarchy), we argue that such two-phase
methods may suffer from error propagation, and cannot effectively optimize
metrics that capture the holistic structure of a taxonomy. In our approach, the
representations of term pairs are learned using multiple sources of information
and used to determine \textit{which} term to select and \textit{where} to place
it on the taxonomy via a policy network. All components are trained in an
end-to-end manner with cumulative rewards, measured by a holistic tree metric
over the training taxonomies. Experiments on two public datasets of different
domains show that our approach outperforms prior state-of-the-art taxonomy
induction methods up to 19.6\% on ancestor F1.Comment: 11 Pages. ACL 2018 Camera Read
Stochastic Coded Federated Learning: Theoretical Analysis and Incentive Mechanism Design
Federated learning (FL) has achieved great success as a privacy-preserving
distributed training paradigm, where many edge devices collaboratively train a
machine learning model by sharing the model updates instead of the raw data
with a server. However, the heterogeneous computational and communication
resources of edge devices give rise to stragglers that significantly decelerate
the training process. To mitigate this issue, we propose a novel FL framework
named stochastic coded federated learning (SCFL) that leverages coded computing
techniques. In SCFL, before the training process starts, each edge device
uploads a privacy-preserving coded dataset to the server, which is generated by
adding Gaussian noise to the projected local dataset. During training, the
server computes gradients on the global coded dataset to compensate for the
missing model updates of the straggling devices. We design a gradient
aggregation scheme to ensure that the aggregated model update is an unbiased
estimate of the desired global update. Moreover, this aggregation scheme
enables periodical model averaging to improve the training efficiency. We
characterize the tradeoff between the convergence performance and privacy
guarantee of SCFL. In particular, a more noisy coded dataset provides stronger
privacy protection for edge devices but results in learning performance
degradation. We further develop a contract-based incentive mechanism to
coordinate such a conflict. The simulation results show that SCFL learns a
better model within the given time and achieves a better privacy-performance
tradeoff than the baseline methods. In addition, the proposed incentive
mechanism grants better training performance than the conventional Stackelberg
game approach
More comprehensive facial inversion for more effective expression recognition
Facial expression recognition (FER) plays a significant role in the
ubiquitous application of computer vision. We revisit this problem with a new
perspective on whether it can acquire useful representations that improve FER
performance in the image generation process, and propose a novel generative
method based on the image inversion mechanism for the FER task, termed
Inversion FER (IFER). Particularly, we devise a novel Adversarial Style
Inversion Transformer (ASIT) towards IFER to comprehensively extract features
of generated facial images. In addition, ASIT is equipped with an image
inversion discriminator that measures the cosine similarity of semantic
features between source and generated images, constrained by a distribution
alignment loss. Finally, we introduce a feature modulation module to fuse the
structural code and latent codes from ASIT for the subsequent FER work. We
extensively evaluate ASIT on facial datasets such as FFHQ and CelebA-HQ,
showing that our approach achieves state-of-the-art facial inversion
performance. IFER also achieves competitive results in facial expression
recognition datasets such as RAF-DB, SFEW and AffectNet. The code and models
are available at https://github.com/Talented-Q/IFER-master
- …