Efficiency Evaluation of Character-level RNN Training Schedules

Abstract

We present four training and prediction schedules from the same character-level recurrent neural network. The efficiency of these schedules is tested in terms of model effectiveness as a function of training time and amount of training data seen. We conclude that the sequence to sequence training, together with sequence to sample prediction, performs the most efficient and consistent across multiple parameter settings. We show that the choice of training and prediction schedule potentially has a considerable impact on the prediction effectiveness for a given training budget