We test the viability of having learnable timescales in multi-timescales recurrent neural networks

Fabien C. Y.  Benureau

Jun Tani

English

We test the viability of having learnable timescales in multi-timescales recurrent neural networks.conference pape

Benureau, Fabien C. Y.

Tani, Jun

OIST Institutional Repository

Learning Timescales in MTRNNs著者（英） Fabien C. Y.  Benureau, Jun Tanijournal orpublication titleThe Proceedings of the 28th Annual Conferenceof the Japanese Neural Network Society page range 177-178year 2018-10-24出版者  Japanese Neural Network Society権利 (C) 2018 Japanese Neural Network SocietyURL http://id.nii.ac.jp/1394/00001272/The 28th Annual Conference of the Japanese Neural Network Society (October, 2018)Learning Timescales in MTRNNsFabien C. Y. Benureau (P)1, and Jun Tani11 Cognitive Neuro-Robotics Unit , Okinawa Institute of Science and TechnologyE-mail: jun.tani@oist.jpAbstract— We test the viability of having learn-able timescales in multi-timescales recurrent neuralnetworks.Keywords— Machine Learning, Recurrent NeuralNetworks, Timescales1 MotivationThis article is quite modest. It seeks to providepreliminary answers to the question: Can and shouldmulti-timescales recurrent neural networks learn theirown timescales? Multi-Timescales Recurrent NeuralNetworks (MTRNN) [1] are recurrent neural networkswith several recurrent layers stacked onto one an-other. Each layer connects to the layer above andbelow, and each layer has its own timescale, whichacts as a decay factor that controls how much thelayer state over the past timesteps influence the cur-rent hidden state. Typically, inspired by the brain, thelower layers—which handle the raw sensory input—have short timescales while the higher layers, that dealwith abstract features, have long timescales and re-act to broad changes in the dynamics of the network.In published MTRNN experiments, timescales valuesare usually set arbitrarily. A popular choice has beenexponential timescales, such as setting the kth layertimescale to 2k (in [2]) or 2k+1 (in [3]).We can wonder, however, if those timescales couldnot be learned during training, like any other connec-tion weight. And if doing so brings any benefits to thelearning performance. In this article, we respond yesto the first question, and remain inconclusive aboutthe second.2 Method2.1 NetworkWe employ a MTRNN [1]. The equations for thelayer k are:hkt+1 =(1− 1τk)hkt +1τk(Ldk−1t + Sdkt +Hdk+1t + b)dkt+1 = tanh(hkt+1)with hkt the hidden state of the layer k at time t, dktthe activity of its units, and L, S, and H the weightmatrices for the connection to the lower layer k−1, thelayer itself, the upper layer k+ 1; and b the bias term.Here, we will consider a network with three layers of25 units each.The time constant τk is the focus of this paperhere. Rather than a fixed scalar, we transform τk ina learnable weight in the network, modified throughback-propagation through time. Moreover, based onour experience, while the learning rate of the rest ofthe network is 0.001, the specific learning rate for thoseτk weights is 0.01, ten times higher. Additionally, thevalue of τk is clamped to a minimum of 1, to avoid adegenerative behavior of the layer equation.2.2 Dataset and TrainingWe consider five sums of one-dimensional sinus pat-terns of 100 timesteps each, as seen on the figure below.The training is done over 20000 epochs, with a0.001 learning rate, and considering a mean-squarederror loss over the dataset patterns. We evaluate thenetwork by its capacity to predict the next timestep,and to reproduce the pattern in closed-loop: the out-put of a timestep is used as input for the next timestep.3 ResultsWe consider a network initialized with all time con-stants at 100 (we pushed the learning rate of the τ to0.1 for this network). Those time constant values aretoo high; they won’t allow the network to generatethe fast variations of the target patterns. As the fig-ure below shows, learning τs during training allow thenetwork to correct for this initialization. The lowestlayer adapts to a low τ value, while the layer 1 and 2remain fairly stable.Note that while in this experiment, the time con-stants ended-up ”in order”, layers do not necessarilyget all ordered when learning τ . If we consider ini-tial τ values 16, 8 and 4 for layer 0, 1 and 2, layer 0again gets a low τ value fast, but layer 2 keeps a lowerτ value than layer 1, at least at epoch 20000, as thefollowing figure shows.The loss of the network benefits from the adapta-tion of the τ values. However, its closed-loop erroractually is worse: the network with fixed τ is mostlyapproximating the slow mean of the samples, whilethe network with learnable τ is starting to learn to re-produce faster features of the patterns, but is not yetgood at it.So: are exponential τ values a good way to ini-tialize a MTRNN? The figure below suggests theymight: τ values of layer 1 and 2 hardly change whenthe network is initialized with exponential τ values.And when initializing all layers at τ = 8, the networkevolves toward the similar state at the exponential ini-tialization.However, when looking at the output of the net-work with τ = 8, those well-differentiated time con-stants do not necessarily help the network at produc-ing the target patterns at epoch 20000, as shown inthe figure that follows.Contrary to our expectations, and while the net-work with learnable τ seems to have settled on goodτ values, the network with a fixed τ = 8 for all threelayers produces better patterns in the closed-loop case.The graph of the closed-loop error shows that the for-mer network is also less stable numerically.4 DiscussionSo let’s review: learning τ in MTRNN works insome cases. It can allow a network to adapt to terribleparameter initialization. It is not clear, however, thatthe values that are derived from this learning processare the best ones for the target patterns. The learningprocess, for instance, consistently hits the hard clampat 1.0 for the lowest layer. If we remove it, the networkwill adopt values of τ as low as 0.6, creating a dynam-ics that will degrade closed-loop learning and stability.Furthermore, it is not clear that the MTRNN perfor-mance is highly dependent on the precise value of itstime constants, as the network with all layers withτ = 8 testifies. One could posit that approximatelygood, fixed time constants might lead to faster andmore stable performance than adaptive ones.This article is exploratory. More complex dataset,simulations and analysis are needed to understand ifand how MTRNN can benefit from learning their timeconstant in an autonomous manner.Code AvailabilityThe source code will be made available for reviewshortly after submission at https://figshare.com/s/d90ec851f08af442627f.References[1] Yamashita, Y., Tani, J. (2008). Emergence of Func-tional Hierarchy in a Multiple Timescale NeuralNetwork Model: A Humanoid Robot Experiment.PLoS Computational Biology, 4(11), e1000220.[2] Choi, M., Tani, J. (2018). Predictive Coding forDynamic Visual Processing: Development of Func-tional Hierarchy in a Multiple SpatiotemporalScales RNN Model. Neural Computation, 30(1),237-270. doi: 10.1162/neco a 01026[3] Ahmadi, A., Tani, J. (2017). Bridging the GapBetween Probabilistic and Deterministic Mod-els: A Simulation Study on a Variational BayesPredictive Coding Recurrent Neural NetworkModel. Neural Information Processing, 760-769.doi: 10.1007/978-3-319-70090-8 77

Learning Timescales in MTRNNs

https://oist.repo.nii.ac.jp/?action=repository_action_common_download&item_id=1403&item_no=1&attribute_id=22&file_no=1

Learning Timescales in MTRNNs

Abstract

Similar works

Full text

Available Versions

OIST Institutional Repository