Bridging the Gap between Probabilistic and Deterministic Models: A Simulation Study on a Variational Bayes Predictive Coding Recurrent Neural Network Model

The current paper proposes a novel variational Bayes predictive coding RNN model, which can learn to generate fluctuated temporal patterns from exemplars. The model learns to maximize the lower bound of the weighted sum of the regularization and reconstruction error terms. We examined how this weighting can affect development of different types of information processing while learning fluctuated temporal patterns. Simulation results show that strong weighting of the reconstruction term causes the development of deterministic chaos for imitating the randomness observed in target sequences, while strong weighting of the regularization term causes the development of stochastic dynamics imitating probabilistic processes observed in targets. Moreover, results indicate that the most generalized learning emerges between these two extremes. The paper concludes with implications in terms of the underlying neuronal mechanisms for autism spectrum disorder and for free action.Comment: This paper is accepted the 24th International Conference On Neural Information Processing (ICONIP 2017). The previous submission to arXiv is replaced by this version because there was an error in Equation

New findings on the prototypical Of?p stars

In recent years several in-depth investigations of the three Galactic Of?p stars were undertaken. These multiwavelength studies revealed the peculiar properties of these objects (in the X-rays as well as in the optical): magnetic fields, periodic line profile variations, recurrent photometric changes. However, many questions remain unsolved. To clarify some of the properties of the Of?p stars, we have continued their monitoring. A new XMM observation and two new optical datasets were obtained. Additional information for the prototypical Of?p trio has been found. HD108 has now reached its quiescent, minimum-emission state, for the first time in 50--60yrs. The echelle spectra of HD148937 confirm the presence of the 7d variations in the Balmer lines and reveal similar periodic variations (though of lower amplitudes) in the HeI5876 and HeII4686 lines, underlining its similarities with the other two prototypical Of?p stars. The new XMM observation of HD191612 was taken at the same phase in the line modulation cycle but at a different orbital phase as previous data. It clearly shows that the X-ray emission of HD191612 is modulated by the 538d period and not the orbital period of 1542d - it is thus not of colliding-wind origin and the phenomenon responsible for the optical changes appears also at work in the high-energy domain. There are however problems: our MHD simulations of the wind magnetic confinement predict both a harder X-ray flux of a much larger strength than what is observed (the modeled DEM peaks at 30-40MK, whereas the observed one peaks at 2MK) and narrow lines (hot gas moving with velocities of 100--200km/s, whereras the observed FWHM is ~2000km/s).Comment: 10 pages, 8 figures (2 in jpg), accepted for publication by A&

Large Margin Neural Language Model

We propose a large margin criterion for training neural language models. Conventionally, neural language models are trained by minimizing perplexity (PPL) on grammatical sentences. However, we demonstrate that PPL may not be the best metric to optimize in some tasks, and further propose a large margin formulation. The proposed method aims to enlarge the margin between the "good" and "bad" sentences in a task-specific sense. It is trained end-to-end and can be widely applied to tasks that involve re-scoring of generated text. Compared with minimum-PPL training, our method gains up to 1.1 WER reduction for speech recognition and 1.0 BLEU increase for machine translation.Comment: 9 pages. Accepted as a long paper in EMNLP201