4,177 research outputs found
Deep Learning with Long Short-Term Memory for Time Series Prediction
Time series prediction can be generalized as a process that extracts useful
information from historical records and then determines future values. Learning
long-range dependencies that are embedded in time series is often an obstacle
for most algorithms, whereas Long Short-Term Memory (LSTM) solutions, as a
specific kind of scheme in deep learning, promise to effectively overcome the
problem. In this article, we first give a brief introduction to the structure
and forward propagation mechanism of the LSTM model. Then, aiming at reducing
the considerable computing cost of LSTM, we put forward the Random Connectivity
LSTM (RCLSTM) model and test it by predicting traffic and user mobility in
telecommunication networks. Compared to LSTM, RCLSTM is formed via stochastic
connectivity between neurons, which achieves a significant breakthrough in the
architecture formation of neural networks. In this way, the RCLSTM model
exhibits a certain level of sparsity, which leads to an appealing decrease in
the computational complexity and makes the RCLSTM model become more applicable
in latency-stringent application scenarios. In the field of telecommunication
networks, the prediction of traffic series and mobility traces could directly
benefit from this improvement as we further demonstrate that the prediction
accuracy of RCLSTM is comparable to that of the conventional LSTM no matter how
we change the number of training samples or the length of input sequences.Comment: 9 pages, 5 figures, 14 reference
Traffic Prediction Based on Random Connectivity in Deep Learning with Long Short-Term Memory
Traffic prediction plays an important role in evaluating the performance of
telecommunication networks and attracts intense research interests. A
significant number of algorithms and models have been put forward to analyse
traffic data and make prediction. In the recent big data era, deep learning has
been exploited to mine the profound information hidden in the data. In
particular, Long Short-Term Memory (LSTM), one kind of Recurrent Neural Network
(RNN) schemes, has attracted a lot of attentions due to its capability of
processing the long-range dependency embedded in the sequential traffic data.
However, LSTM has considerable computational cost, which can not be tolerated
in tasks with stringent latency requirement. In this paper, we propose a deep
learning model based on LSTM, called Random Connectivity LSTM (RCLSTM).
Compared to the conventional LSTM, RCLSTM makes a notable breakthrough in the
formation of neural network, which is that the neurons are connected in a
stochastic manner rather than full connected. So, the RCLSTM, with certain
intrinsic sparsity, have many neural connections absent (distinguished from the
full connectivity) and which leads to the reduction of the parameters to be
trained and the computational cost. We apply the RCLSTM to predict traffic and
validate that the RCLSTM with even 35% neural connectivity still shows a
satisfactory performance. When we gradually add training samples, the
performance of RCLSTM becomes increasingly closer to the baseline LSTM.
Moreover, for the input traffic sequences of enough length, the RCLSTM exhibits
even superior prediction accuracy than the baseline LSTM.Comment: 6 pages, 9 figure
Multi-task additive models with shared transfer functions based on dictionary learning
Additive models form a widely popular class of regression models which
represent the relation between covariates and response variables as the sum of
low-dimensional transfer functions. Besides flexibility and accuracy, a key
benefit of these models is their interpretability: the transfer functions
provide visual means for inspecting the models and identifying domain-specific
relations between inputs and outputs. However, in large-scale problems
involving the prediction of many related tasks, learning independently additive
models results in a loss of model interpretability, and can cause overfitting
when training data is scarce. We introduce a novel multi-task learning approach
which provides a corpus of accurate and interpretable additive models for a
large number of related forecasting tasks. Our key idea is to share transfer
functions across models in order to reduce the model complexity and ease the
exploration of the corpus. We establish a connection with sparse dictionary
learning and propose a new efficient fitting algorithm which alternates between
sparse coding and transfer function updates. The former step is solved via an
extension of Orthogonal Matching Pursuit, whose properties are analyzed using a
novel recovery condition which extends existing results in the literature. The
latter step is addressed using a traditional dictionary update rule.
Experiments on real-world data demonstrate that our approach compares favorably
to baseline methods while yielding an interpretable corpus of models, revealing
structure among the individual tasks and being more robust when training data
is scarce. Our framework therefore extends the well-known benefits of additive
models to common regression settings possibly involving thousands of tasks
European exchange trading funds trading with locally weighted support vector regression
In this paper, two different Locally Weighted Support Vector Regression (wSVR) algorithms are generated and applied to the task of forecasting and trading five European Exchange Traded Funds. The trading application covers the recent European Monetary Union debt crisis. The performance of the proposed models is benchmarked against traditional Support Vector Regression (SVR) models. The Radial Basis Function, the Wavelet and the Mahalanobis kernel are explored and tested as SVR kernels. Finally, a novel statistical SVR input selection procedure is introduced based on a principal component analysis and the Hansen, Lunde, and Nason (2011) model confidence test. The results demonstrate the superiority of the wSVR models over the traditional SVRs and of the v-SVR over the ε-SVR algorithms. We note that the performance of all models varies and considerably deteriorates in the peak of the debt crisis. In terms of the kernels, our results do not confirm the belief that the Radial Basis Function is the optimum choice for financial series
- …