1 research outputs found
L1 regularization is better than L2 for learning and predicting chaotic systems
Emergent behaviors are in the focus of recent research interest. It is then
of considerable importance to investigate what optimizations suit the learning
and prediction of chaotic systems, the putative candidates for emergence. We
have compared L1 and L2 regularizations on predicting chaotic time series using
linear recurrent neural networks. The internal representation and the weights
of the networks were optimized in a unifying framework. Computational tests on
different problems indicate considerable advantages for the L1 regularization:
It had considerably better learning time and better interpolating capabilities.
We shall argue that optimization viewed as a maximum likelihood estimation
justifies our results, because L1 regularization fits heavy-tailed
distributions -- an apparently general feature of emergent systems -- better.Comment: 13 pages, 4 figure