Low-bit Shift Network for End-to-End Spoken Language Understanding

Deep neural networks (DNN) have achieved impressive success in multiple domains. Over the years, the accuracy of these models has increased with the proliferation of deeper and more complex architectures. Thus, state-of-the-art solutions are often computationally expensive, which makes them unfit to be deployed on edge computing platforms. In order to mitigate the high computation, memory, and power requirements of inferring convolutional neural networks (CNNs), we propose the use of power-of-two quantization, which quantizes continuous parameters into low-bit power-of-two values. This reduces computational complexity by removing expensive multiplication operations and with the use of low-bit weights. ResNet is adopted as the building block of our solution and the proposed model is evaluated on a spoken language understanding (SLU) task. Experimental results show improved performance for shift neural network architectures, with our low-bit quantization achieving 98.76 \% on the test set which is comparable performance to its full-precision counterpart and state-of-the-art solutions.Comment: Accepted at INTERSPEECH 202

Traffic Scheduling Strategy of Power Communication Network Based on SDN

Due to the complicated structure, power communication network is difficult to guarantee the quality of service (QoS) of power services. A two-level scheduling algorithm based on software defined network (SDN) is proposed in this paper. Firstly, the priority-based scheduling method is used to meet the latency-sensitive of power service. Then, in order to alleviate congestion, queue bandwidth is adjusted according to network state information, which can be collected by the centralized control of SDN. Finally, the Mininet and Ryu controller are made use of building simulation environment. The test results show that the algorithm proposed in this paper reduce delay and packet loss rate significantly, which achieves QoS

DenseShift: Towards Accurate and Efficient Low-Bit Power-of-Two Quantization

Efficiently deploying deep neural networks on low-resource edge devices is challenging due to their ever-increasing resource requirements. To address this issue, researchers have proposed multiplication-free neural networks, such as Power-of-Two quantization, or also known as Shift networks, which aim to reduce memory usage and simplify computation. However, existing low-bit Shift networks are not as accurate as their full-precision counterparts, typically suffering from limited weight range encoding schemes and quantization loss. In this paper, we propose the DenseShift network, which significantly improves the accuracy of Shift networks, achieving competitive performance to full-precision networks for vision and speech applications. In addition, we introduce a method to deploy an efficient DenseShift network using non-quantized floating-point activations, while obtaining 1.6X speed-up over existing methods. To achieve this, we demonstrate that zero-weight values in low-bit Shift networks do not contribute to model capacity and negatively impact inference computation. To address this issue, we propose a zero-free shifting mechanism that simplifies inference and increases model capacity. We further propose a sign-scale decomposition design to enhance training efficiency and a low-variance random initialization strategy to improve the model's transfer learning performance. Our extensive experiments on various computer vision and speech tasks demonstrate that DenseShift outperforms existing low-bit multiplication-free networks and achieves competitive performance compared to full-precision networks. Furthermore, our proposed approach exhibits strong transfer learning performance without a drop in accuracy. Our code was released on GitHub

Extracting the Quantum Geometric Tensor of an Optical Raman Lattice by Bloch State Tomography

In Hilbert space, the geometry of the quantum state is identified by the quantum geometric tensor (QGT), whose imaginary part is the Berry curvature and real part is the quantum metric tensor. Here, we propose and experimentally implement a complete Bloch state tomography to directly measure eigenfunction of an optical Raman lattice for ultracold atoms. Through the measured eigenfunction, the distribution of the complete QGT in the Brillouin zone is reconstructed, with which the topological invariants are extracted by the Berry curvature and the distances of quantum states in momentum space are measured by the quantum metric tensor. Further, we experimentally test a predicted inequality between the Berry curvature and quantum metric tensor, which reveals a deep connection between topology and geometry