17 research outputs found
Traffic Prediction Based on Random Connectivity in Deep Learning with Long Short-Term Memory
Traffic prediction plays an important role in evaluating the performance of
telecommunication networks and attracts intense research interests. A
significant number of algorithms and models have been put forward to analyse
traffic data and make prediction. In the recent big data era, deep learning has
been exploited to mine the profound information hidden in the data. In
particular, Long Short-Term Memory (LSTM), one kind of Recurrent Neural Network
(RNN) schemes, has attracted a lot of attentions due to its capability of
processing the long-range dependency embedded in the sequential traffic data.
However, LSTM has considerable computational cost, which can not be tolerated
in tasks with stringent latency requirement. In this paper, we propose a deep
learning model based on LSTM, called Random Connectivity LSTM (RCLSTM).
Compared to the conventional LSTM, RCLSTM makes a notable breakthrough in the
formation of neural network, which is that the neurons are connected in a
stochastic manner rather than full connected. So, the RCLSTM, with certain
intrinsic sparsity, have many neural connections absent (distinguished from the
full connectivity) and which leads to the reduction of the parameters to be
trained and the computational cost. We apply the RCLSTM to predict traffic and
validate that the RCLSTM with even 35% neural connectivity still shows a
satisfactory performance. When we gradually add training samples, the
performance of RCLSTM becomes increasingly closer to the baseline LSTM.
Moreover, for the input traffic sequences of enough length, the RCLSTM exhibits
even superior prediction accuracy than the baseline LSTM.Comment: 6 pages, 9 figure
An Approximate Bayesian Long Short-Term Memory Algorithm for Outlier Detection
Long Short-Term Memory networks trained with gradient descent and
back-propagation have received great success in various applications. However,
point estimation of the weights of the networks is prone to over-fitting
problems and lacks important uncertainty information associated with the
estimation. However, exact Bayesian neural network methods are intractable and
non-applicable for real-world applications. In this study, we propose an
approximate estimation of the weights uncertainty using Ensemble Kalman Filter,
which is easily scalable to a large number of weights. Furthermore, we optimize
the covariance of the noise distribution in the ensemble update step using
maximum likelihood estimation. To assess the proposed algorithm, we apply it to
outlier detection in five real-world events retrieved from the Twitter
platform