Compressing Recurrent Neural Networks with Tensor Ring for Action Recognition

Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved promising performance in sequential data modeling. The hidden layers in RNNs can be regarded as the memory units, which are helpful in storing information in sequential contexts. However, when dealing with high dimensional input data, such as video and text, the input-to-hidden linear transformation in RNNs brings high memory usage and huge computational cost. This makes the training of RNNs unscalable and difficult. To address this challenge, we propose a novel compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring decomposition (TRD) to reformulate the input-to-hidden transformation. Compared with other tensor decomposition methods, TR-LSTM is more stable. In addition, TR-LSTM can complete an end-to-end training and also provide a fundamental building block for RNNs in handling large input data. Experiments on real-world action recognition datasets have demonstrated the promising performance of the proposed TR-LSTM compared with the tensor train LSTM and other state-of-the-art competitors.Comment: 9 page

Valid Physical Processes from Numerical Discontinuities in Computational Fluid Dynamics

Due to the limited cell resolution in the representation of flow variables, a piecewise continuous initial reconstruction with discontinuous jump at a cell interface is usually used in modern computational fluid dynamics methods. Starting from the discontinuity, a Riemann problem in the Godunov method is solved for the flux evaluation across the cell interface in a finite volume scheme. With the increasing of Mach number in the CFD simulations, the adaptation of the Riemann solver seems introduce intrinsically a mechanism to develop instabilities in strong shock regions. Theoretically, the Riemann solution of the Euler equations are based on the equilibrium assumption, which may not be valid in the non-equilibrium shock layer. In order to clarify the flow physics from a discontinuity, the unsteady flow behavior of one-dimensional contact and shock wave is studied on a time scale of (0~10000) times of the particle collision time. In the study of the non-equilibrium flow behavior from a discontinuity, the collision-less Boltzmann equation is first used for the time scale within one particle collision time, then the direct simulation Monte Carlo (DSMC) method will be adapted to get the further evolution solution. The transition from the free particle transport to the dissipative Navier-Stokes (NS) solutions are obtained as an increasing of time. The exact Riemann solution becomes a limiting solution with infinite number of particle collisions. For the high Mach number flow simulations, the points in the shock transition region, even though the region is enlarged numerically to the mesh size, should be considered as the points inside a highly non-equilibrium shock layer

Exploiting Rich Syntactic Information for Semantic Parsing with Graph-to-Sequence Model

Existing neural semantic parsers mainly utilize a sequence encoder, i.e., a sequential LSTM, to extract word order features while neglecting other valuable syntactic information such as dependency graph or constituent trees. In this paper, we first propose to use the \textit{syntactic graph} to represent three types of syntactic information, i.e., word order, dependency and constituency features. We further employ a graph-to-sequence model to encode the syntactic graph and decode a logical form. Experimental results on benchmark datasets show that our model is comparable to the state-of-the-art on Jobs640, ATIS and Geo880. Experimental results on adversarial examples demonstrate the robustness of the model is also improved by encoding more syntactic information.Comment: EMNLP'1

Intelligent optical performance monitor using multi-task learning based artificial neural network

An intelligent optical performance monitor using multi-task learning based artificial neural network (MTL-ANN) is designed for simultaneous OSNR monitoring and modulation format identification (MFI). Signals' amplitude histograms (AHs) after constant module algorithm are selected as the input features for MTL-ANN. The experimental results of 20-Gbaud NRZ-OOK, PAM4 and PAM8 signals demonstrate that MTL-ANN could achieve OSNR monitoring and MFI simultaneously with higher accuracy and stability compared with single-task learning based ANNs (STL-ANNs). The results show an MFI accuracy of 100% and OSNR monitoring root-mean-square error of 0.63 dB for the three modulation formats under consideration. Furthermore, the number of neuron needed for the single MTL-ANN is almost the half of STL-ANN, which enables reduced-complexity optical performance monitoring devices for real-time performance monitoring

Abnormal traffic detection system in SDN based on deep learning hybrid models

Software defined network (SDN) provides technical support for network construction in smart cities, However, the openness of SDN is also prone to more network attacks. Traditional abnormal traffic detection methods have complex algorithms and find it difficult to detect abnormalities in the network promptly, which cannot meet the demand for abnormal detection in the SDN environment. Therefore, we propose an abnormal traffic detection system based on deep learning hybrid model. The system adopts a hierarchical detection technique, which first achieves rough detection of abnormal traffic based on port information. Then it uses wavelet transform and deep learning techniques for fine detection of all traffic data flowing through suspicious switches. The experimental results show that the proposed detection method based on port information can quickly complete the approximate localization of the source of abnormal traffic. the accuracy, precision, and recall of the fine detection are significantly improved compared with the traditional method of abnormal traffic detection in SDN

Phycocyanin relieves myocardial ischemia-reperfusion injury in rats by inhibiting oxidative stress

Purpose: To investigate the effect of phycocyanin on myocardial ischemia-reperfusion injury, and the possible mechanisms involved. Methods: Twenty-four Sprague-Dawley (SD) rats were randomly divided into Sham group (only threading without ligation), IRI group (myocardial ischemia-reperfusion injury group) and phycocyanin group (phycocyanin pretreatment + myocardial ischemia-reperfusion injury group). The heart was harvested and cardiomyocytes were isolated. Colorimetry was used to determine the contents of cardiomyocyte serum creatine phospho-MB (CK-MB), lactate dehydrogenase (LDH) and malondialdehyde (MDA), and the activities of total antioxidant capacity (T-AOC), catalase (CAT), glutathione (GSH), total superoxide dismutase (SOD) and other related oxidative stress indicators. Furthermore, apoptosis was evaluated using TUNEL staining. Protein levels of cardiac factor E2 related factor 2 (Nrf2), heme oxygenase-1 (HO-1), human NADPH dehydrogenase 1 (NQO1) and nuclear factor-κB (NF-κB) were evaluated by Western blot and immunohistochemistry. Results: Compared with the myocardial IRI group, the contents of CK-MB, LDH, MAD and ROS in the treated group were significantly decreased (p < 0.05), but the activities of SOD, GSH, SOD, CAT, and T-AOC in the myocardial tissues were significantly enhanced (p < 0.05). Moreover, the pathological changes in myocardial tissue were significantly reduced. In addition, the expression levels of Nrf2, HO-1 and NQO-1 were significantly up-regulated after phycocyanin pretreatment, while expression of NF-κB was significantly down-regulated (p < 0.05). Conclusion: Phycocyanin improves myocardial anti-oxidative stress via activation of Nrf2 signaling pathway, and also protects rats from myocardial ischemia-reperfusion injury by reducing inflammatory response via inhibition of NF-κB signaling pathway