6,317 research outputs found
Preserving Node-level Privacy in Graph Neural Networks
Differential privacy (DP) has seen immense applications in learning on
tabular, image, and sequential data where instance-level privacy is concerned.
In learning on graphs, contrastingly, works on node-level privacy are highly
sparse. Challenges arise as existing DP protocols hardly apply to the
message-passing mechanism in Graph Neural Networks (GNNs).
In this study, we propose a solution that specifically addresses the issue of
node-level privacy. Our protocol consists of two main components: 1) a sampling
routine called HeterPoisson, which employs a specialized node sampling strategy
and a series of tailored operations to generate a batch of sub-graphs with
desired properties, and 2) a randomization routine that utilizes symmetric
multivariate Laplace (SML) noise instead of the commonly used Gaussian noise.
Our privacy accounting shows this particular combination provides a non-trivial
privacy guarantee. In addition, our protocol enables GNN learning with good
performance, as demonstrated by experiments on five real-world datasets;
compared with existing baselines, our method shows significant advantages,
especially in the high privacy regime. Experimentally, we also 1) perform
membership inference attacks against our protocol and 2) apply privacy audit
techniques to confirm our protocol's privacy integrity.
In the sequel, we present a study on a seemingly appealing approach
\cite{sajadmanesh2023gap} (USENIX'23) that protects node-level privacy via
differentially private node/instance embeddings. Unfortunately, such work has
fundamental privacy flaws, which are identified through a thorough case study.
More importantly, we prove an impossibility result of achieving both (strong)
privacy and (acceptable) utility through private instance embedding. The
implication is that such an approach has intrinsic utility barriers when
enforcing differential privacy
Practical Differentially Private and Byzantine-resilient Federated Learning
Privacy and Byzantine resilience are two indispensable requirements for a
federated learning (FL) system. Although there have been extensive studies on
privacy and Byzantine security in their own track, solutions that consider both
remain sparse. This is due to difficulties in reconciling privacy-preserving
and Byzantine-resilient algorithms.
In this work, we propose a solution to such a two-fold issue. We use our
version of differentially private stochastic gradient descent (DP-SGD)
algorithm to preserve privacy and then apply our Byzantine-resilient
algorithms. We note that while existing works follow this general approach, an
in-depth analysis on the interplay between DP and Byzantine resilience has been
ignored, leading to unsatisfactory performance. Specifically, for the random
noise introduced by DP, previous works strive to reduce its impact on the
Byzantine aggregation. In contrast, we leverage the random noise to construct
an aggregation that effectively rejects many existing Byzantine attacks.
We provide both theoretical proof and empirical experiments to show our
protocol is effective: retaining high accuracy while preserving the DP
guarantee and Byzantine resilience. Compared with the previous work, our
protocol 1) achieves significantly higher accuracy even in a high privacy
regime; 2) works well even when up to 90% of distributive workers are
Byzantine
Ambient Sound Helps: Audiovisual Crowd Counting in Extreme Conditions
Visual crowd counting has been recently studied as a way to enable people
counting in crowd scenes from images. Albeit successful, vision-based crowd
counting approaches could fail to capture informative features in extreme
conditions, e.g., imaging at night and occlusion. In this work, we introduce a
novel task of audiovisual crowd counting, in which visual and auditory
information are integrated for counting purposes. We collect a large-scale
benchmark, named auDiovISual Crowd cOunting (DISCO) dataset, consisting of
1,935 images and the corresponding audio clips, and 170,270 annotated
instances. In order to fuse the two modalities, we make use of a linear
feature-wise fusion module that carries out an affine transformation on visual
and auditory features. Finally, we conduct extensive experiments using the
proposed dataset and approach. Experimental results show that introducing
auditory information can benefit crowd counting under different illumination,
noise, and occlusion conditions. The dataset and code will be released. Code
and data have been made availabl
Denoising Diffusion Autoencoders are Unified Self-supervised Learners
Inspired by recent advances in diffusion models, which are reminiscent of
denoising autoencoders, we investigate whether they can acquire discriminative
representations for classification via generative pre-training. This paper
shows that the networks in diffusion models, namely denoising diffusion
autoencoders (DDAE), are unified self-supervised learners: by pre-training on
unconditional image generation, DDAE has already learned strongly
linear-separable representations within its intermediate layers without
auxiliary encoders, thus making diffusion pre-training emerge as a general
approach for generative-and-discriminative dual learning. To validate this, we
conduct linear probe and fine-tuning evaluations. Our diffusion-based approach
achieves 95.9% and 50.0% linear evaluation accuracies on CIFAR-10 and
Tiny-ImageNet, respectively, and is comparable to contrastive learning and
masked autoencoders for the first time. Transfer learning from ImageNet also
confirms the suitability of DDAE for Vision Transformers, suggesting the
potential to scale DDAEs as unified foundation models. Code is available at
github.com/FutureXiang/ddae.Comment: ICCV 2023 Ora
Performance of Spatial Modulation using Measured Real-World Channels
In this paper, for the first time real-world channel measurements are used to
analyse the performance of spatial modulation (SM), where a full analysis of
the average bit error rate performance (ABER) of SM using measured urban
correlated and uncorrelated Rayleigh fading channels is provided. The channel
measurements are taken from an outdoor urban multiple input multiple output
(MIMO) measurement campaign. Moreover, ABER performance results using simulated
Rayleigh fading channels are provided and compared with a derived analytical
bound for the ABER of SM, and the ABER results for SM using the measured urban
channels. The ABER results using the measured urban channels validate the
derived analytical bound and the ABER results using the simulated channels.
Finally, the ABER of SM is compared with the performance of spatial
multiplexing (SMX) using the measured urban channels for small and large scale
MIMO. It is shown that SM offers nearly the same or a slightly better
performance than SMX for small scale MIMO. However, SM offers large reduction
in ABER for large scale MIMO.Comment: IEEE Vehicular Technology Conference Fall 2013 (VTC-Fall 2013),
Accepte
- …