6 research outputs found
SPEC2: SPECtral SParsE CNN Accelerator on FPGAs
To accelerate inference of Convolutional Neural Networks (CNNs), various
techniques have been proposed to reduce computation redundancy. Converting
convolutional layers into frequency domain significantly reduces the
computation complexity of the sliding window operations in space domain. On the
other hand, weight pruning techniques address the redundancy in model
parameters by converting dense convolutional kernels into sparse ones. To
obtain high-throughput FPGA implementation, we propose SPEC2 -- the first work
to prune and accelerate spectral CNNs. First, we propose a systematic pruning
algorithm based on Alternative Direction Method of Multipliers (ADMM). The
offline pruning iteratively sets the majority of spectral weights to zero,
without using any handcrafted heuristics. Then, we design an optimized pipeline
architecture on FPGA that has efficient random access into the sparse kernels
and exploits various dimensions of parallelism in convolutional layers.
Overall, SPEC2 achieves high inference throughput with extremely low
computation complexity and negligible accuracy degradation. We demonstrate
SPEC2 by pruning and implementing LeNet and VGG16 on the Xilinx Virtex
platform. After pruning 75% of the spectral weights, SPEC2 achieves 0% accuracy
loss for LeNet, and <1% accuracy loss for VGG16. The resulting accelerators
achieve up to 24x higher throughput, compared with the state-of-the-art FPGA
implementations for VGG16.Comment: This is a 10-page conference paper in 26TH IEEE International
Conference On High Performance Computing, Data, and Analytics (HiPC
SCE: Scalable Network Embedding from Sparsest Cut
Large-scale network embedding is to learn a latent representation for each
node in an unsupervised manner, which captures inherent properties and
structural information of the underlying graph. In this field, many popular
approaches are influenced by the skip-gram model from natural language
processing. Most of them use a contrastive objective to train an encoder which
forces the embeddings of similar pairs to be close and embeddings of negative
samples to be far. A key of success to such contrastive learning methods is how
to draw positive and negative samples. While negative samples that are
generated by straightforward random sampling are often satisfying, methods for
drawing positive examples remains a hot topic.
In this paper, we propose SCE for unsupervised network embedding only using
negative samples for training. Our method is based on a new contrastive
objective inspired by the well-known sparsest cut problem. To solve the
underlying optimization problem, we introduce a Laplacian smoothing trick,
which uses graph convolutional operators as low-pass filters for smoothing node
representations. The resulting model consists of a GCN-type structure as the
encoder and a simple loss function. Notably, our model does not use positive
samples but only negative samples for training, which not only makes the
implementation and tuning much easier, but also reduces the training time
significantly.
Finally, extensive experimental studies on real world data sets are
conducted. The results clearly demonstrate the advantages of our new model in
both accuracy and scalability compared to strong baselines such as GraphSAGE,
G2G and DGI.Comment: KDD 202