55 research outputs found
Deep Learning-Based Frequency Offset Estimation
In wireless communication systems, the asynchronization of the oscillators in
the transmitter and the receiver along with the Doppler shift due to relative
movement may lead to the presence of carrier frequency offset (CFO) in the
received signals. Estimation of CFO is crucial for subsequent processing such
as coherent demodulation. In this brief, we demonstrate the utilization of deep
learning for CFO estimation by employing a residual network (ResNet) to learn
and extract signal features from the raw in-phase (I) and quadrature (Q)
components of the signals. We use multiple modulation schemes in the training
set to make the trained model adaptable to multiple modulations or even new
signals. In comparison to the commonly used traditional CFO estimation methods,
our proposed IQ-ResNet method exhibits superior performance across various
scenarios including different oversampling ratios, various signal lengths, and
different channel
Graph-Based Similarity of Neural Network Representations
Understanding the black-box representations in Deep Neural Networks (DNN) is
an essential problem in deep learning. In this work, we propose Graph-Based
Similarity (GBS) to measure the similarity of layer features. Contrary to
previous works that compute the similarity directly on the feature maps, GBS
measures the correlation based on the graph constructed with hidden layer
outputs. By treating each input sample as a node and the corresponding layer
output similarity as edges, we construct the graph of DNN representations for
each layer. The similarity between graphs of layers identifies the
correspondences between representations of models trained in different datasets
and initializations. We demonstrate and prove the invariance property of GBS,
including invariance to orthogonal transformation and invariance to isotropic
scaling, and compare GBS with CKA. GBS shows state-of-the-art performance in
reflecting the similarity and provides insights on explaining the adversarial
sample behavior on the hidden layer space
Subgraph Networks Based Contrastive Learning
Graph contrastive learning (GCL), as a self-supervised learning method, can
solve the problem of annotated data scarcity. It mines explicit features in
unannotated graphs to generate favorable graph representations for downstream
tasks. Most existing GCL methods focus on the design of graph augmentation
strategies and mutual information estimation operations. Graph augmentation
produces augmented views by graph perturbations. These views preserve a locally
similar structure and exploit explicit features. However, these methods have
not considered the interaction existing in subgraphs. To explore the impact of
substructure interactions on graph representations, we propose a novel
framework called subgraph network-based contrastive learning (SGNCL). SGNCL
applies a subgraph network generation strategy to produce augmented views. This
strategy converts the original graph into an Edge-to-Node mapping network with
both topological and attribute features. The single-shot augmented view is a
first-order subgraph network that mines the interaction between nodes,
node-edge, and edges. In addition, we also investigate the impact of the
second-order subgraph augmentation on mining graph structure interactions, and
further, propose a contrastive objective that fuses the first-order and
second-order subgraph information. We compare SGNCL with classical and
state-of-the-art graph contrastive learning methods on multiple benchmark
datasets of different domains. Extensive experiments show that SGNCL achieves
competitive or better performance (top three) on all datasets in unsupervised
learning settings. Furthermore, SGNCL achieves the best average gain of 6.9\%
in transfer learning compared to the best method. Finally, experiments also
demonstrate that mining substructure interactions have positive implications
for graph contrastive learning.Comment: 12 pages, 6 figure
Understanding the Dynamics of DNNs Using Graph Modularity
There are good arguments to support the claim that deep neural networks
(DNNs) capture better feature representations than the previous hand-crafted
feature engineering, which leads to a significant performance improvement. In
this paper, we move a tiny step towards understanding the dynamics of feature
representations over layers. Specifically, we model the process of class
separation of intermediate representations in pre-trained DNNs as the evolution
of communities in dynamic graphs. Then, we introduce modularity, a generic
metric in graph theory, to quantify the evolution of communities. In the
preliminary experiment, we find that modularity roughly tends to increase as
the layer goes deeper and the degradation and plateau arise when the model
complexity is great relative to the dataset. Through an asymptotic analysis, we
prove that modularity can be broadly used for different applications. For
example, modularity provides new insights to quantify the difference between
feature representations. More crucially, we demonstrate that the degradation
and plateau in modularity curves represent redundant layers in DNNs and can be
pruned with minimal impact on performance, which provides theoretical guidance
for layer pruning. Our code is available at
https://github.com/yaolu-zjut/Dynamic-Graphs-Construction.Comment: Accepted by ECCV 202
- …