50,457 research outputs found
Echo State Networks with Self-Normalizing Activations on the Hyper-Sphere
Among the various architectures of Recurrent Neural Networks, Echo State
Networks (ESNs) emerged due to their simplified and inexpensive training
procedure. These networks are known to be sensitive to the setting of
hyper-parameters, which critically affect their behaviour. Results show that
their performance is usually maximized in a narrow region of hyper-parameter
space called edge of chaos. Finding such a region requires searching in
hyper-parameter space in a sensible way: hyper-parameter configurations
marginally outside such a region might yield networks exhibiting fully
developed chaos, hence producing unreliable computations. The performance gain
due to optimizing hyper-parameters can be studied by considering the
memory--nonlinearity trade-off, i.e., the fact that increasing the nonlinear
behavior of the network degrades its ability to remember past inputs, and
vice-versa. In this paper, we propose a model of ESNs that eliminates critical
dependence on hyper-parameters, resulting in networks that provably cannot
enter a chaotic regime and, at the same time, denotes nonlinear behaviour in
phase space characterised by a large memory of past inputs, comparable to the
one of linear networks. Our contribution is supported by experiments
corroborating our theoretical findings, showing that the proposed model
displays dynamics that are rich-enough to approximate many common nonlinear
systems used for benchmarking
Recurrent Neural Networks for Online Video Popularity Prediction
In this paper, we address the problem of popularity prediction of online
videos shared in social media. We prove that this challenging task can be
approached using recently proposed deep neural network architectures. We cast
the popularity prediction problem as a classification task and we aim to solve
it using only visual cues extracted from videos. To that end, we propose a new
method based on a Long-term Recurrent Convolutional Network (LRCN) that
incorporates the sequentiality of the information in the model. Results
obtained on a dataset of over 37'000 videos published on Facebook show that
using our method leads to over 30% improvement in prediction performance over
the traditional shallow approaches and can provide valuable insights for
content creators
- …