Deep Gate Recurrent Neural Network

Abstract This paper explores the possibility of using multiplicative gate to build two recurrent neural network structures. These two structures are called Deep Simple Gated Unit (DSGU) and Simple Gated Unit (SGU), which are structures for learning long-term dependencies. Compared to traditional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), both structures require fewer parameters and less computation time in sequence classification tasks. Unlike GRU and LSTM, which require more than one gate to control information flow in the network, SGU and DSGU only use one multiplicative gate to control the flow of information. We show that this difference can accelerate the learning speed in tasks that require long dependency information. We also show that DSGU is more numerically stable than SGU. In addition, we also propose a standard way of representing the inner structure of RNN called RNN Conventional Graph (RCG), which helps to analyze the relationship between input units and hidden units of RNN

Deep Gate Recurrent Neural Network

This paper introduces two recurrent neural network structures called Simple Gated Unit (SGU) and Deep Simple Gated Unit (DSGU), which are general structures for learning long term dependencies. Compared to traditional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), both structures require fewer parameters and less computation time in sequence classification tasks. Unlike GRU and LSTM, which require more than one gates to control information flow in the network, SGU and DSGU only use one multiplicative gate to control the flow of information. We show that this difference can accelerate the learning speed in tasks that require long dependency information. We also show that DSGU is more numerically stable than SGU. In addition, we also propose a standard way of representing inner structure of RNN called RNN Conventional Graph (RCG), which helps analyzing the relationship between input units and hidden units of RNN.Comment: This paper has been withdrawn by the author due to lacking of enough experiment

Smart IoT Network Based Convolutional Recurrent Neural Network with Element-Wise Prediction System

© Copyright 2021, The Author(s). An Intelligent Internet of Things network based on an Artificial Intelligent System, can substantially control and reduce the congestion effects in the network. In this paper, an artificial intelligent system is proposed for eliminating the congestion effects in traffic load in an Intelligent Internet of Things network based on a deep learning Convolutional Recurrent Neural Network with a modified Element-wise Attention Gate. The invisible layer of the modified Element-wise Attention Gate structure has self-feedback to increase its long short-term memory. The artificial intelligent system is implemented for next step ahead traffic estimation and clustering the network. In the proposed architecture, each sensing node is adaptive and able to change its affiliation with other clusters based on a deep learning modified Element-wise Attention Gate. The modified Element-wise Attention Gate has the ability to handle the buffer capacity in all the network, thereby enriching the Quality of Service. A deep learning modified training algorithm is proposed to learn the artificial intelligent system allowing the neurons to have greater concentration ability. The simulation results demonstrate that the Root Mean Square error is minimized by 37.14% when using modified Element-wise Attention Gate when compared with a Deep Learning Recurrent Neural Network. Also, the Quality of Service of the network is improved, for example, the network lifetime is enhanced by 12.7% more than with Deep Learning Recurrent Neural Network

Cyclic gate recurrent neural networks for time series data with missing values

Gated Recurrent Neural Networks (RNNs) such as LSTM and GRU have been highly effective in handling sequential time series data in recent years. Although Gated RNNs have an inherent ability to learn complex temporal dynamics, there is potential for further enhancement by enabling these deep learning networks to directly use time information to recognise time-dependent patterns in data and identify important segments of time. Synonymous with time series data in real-world applications are missing values, which often reduce a model’s ability to perform predictive tasks. Historically, missing values have been handled by simple or complex imputation techniques as well as machine learning models, which manage the missing values in the prediction layers. However, these methods do not attempt to identify the significance of data segments and therefore are susceptible to poor imputation values or model degradation from high missing value rates. This paper develops Cyclic Gate enhanced recurrent neural networks with learnt waveform parameters to automatically identify important data segments within a time series and neglect unimportant segments. By using the proposed networks, the negative impact of missing data on model performance is mitigated through the addition of customised cyclic opening and closing gate operations. Cyclic Gate Recurrent Neural Networks are tested on several sequential time series datasets for classification performance. For long sequence datasets with high rates of missing values, Cyclic Gate enhanced RNN models achieve higher performance metrics than standard gated recurrent neural network models, conventional non-neural network machine learning algorithms and current state of the art RNN cell variants

Modelos de clasificación binaria de la coloración semántica de textos

Introduction: The purpose of the research is to compare different types of recurrent neural network architectures, namely the long short-term memory and gate recurrent node architecture and the convolutional neural network, and to explore their performance on the example of binary text classification. Material and Methods: To achieve this, the research evaluates the performance of these two popular deep-learning approaches on a dataset consisting of film reviews that are marked with both positive and adverse opinions. The real-world dataset was used to train neural network models using software implementations. Results and Discussion: The research focuses on the implementation of a recurrent neural network for the binary classification of a dataset and explores different types of architecture, approaches and hyperparameters to determine the best model to achieve optimal performance. The software implementation allowed evaluating of various quality metrics, which allowed comparing the performance of the proposed approaches. In addition, the research explores various hyperparameters such as learning rate, packet sizes, and regulation methods to determine their impact on model performance. Conclusion: In general, the research provides valuable insights into the performance of neural networks in binary text classification and highlights the importance of careful architecture selection and hyperparameter tuning to achieve optimal performance