688 research outputs found
Long-Range Correlation Underlying Childhood Language and Generative Models
Long-range correlation, a property of time series exhibiting long-term
memory, is mainly studied in the statistical physics domain and has been
reported to exist in natural language. Using a state-of-the-art method for such
analysis, long-range correlation is first shown to occur in long CHILDES data
sets. To understand why, Bayesian generative models of language, originally
proposed in the cognitive scientific domain, are investigated. Among
representative models, the Simon model was found to exhibit surprisingly good
long-range correlation, but not the Pitman-Yor model. Since the Simon model is
known not to correctly reflect the vocabulary growth of natural language, a
simple new model is devised as a conjunct of the Simon and Pitman-Yor models,
such that long-range correlation holds with a correct vocabulary growth rate.
The investigation overall suggests that uniform sampling is one cause of
long-range correlation and could thus have a relation with actual linguistic
processes
Do Neural Nets Learn Statistical Laws behind Natural Language?
The performance of deep learning in natural language processing has been
spectacular, but the reasons for this success remain unclear because of the
inherent complexity of deep learning. This paper provides empirical evidence of
its effectiveness and of a limitation of neural networks for language
engineering. Precisely, we demonstrate that a neural language model based on
long short-term memory (LSTM) effectively reproduces Zipf's law and Heaps' law,
two representative statistical properties underlying natural language. We
discuss the quality of reproducibility and the emergence of Zipf's law and
Heaps' law as training progresses. We also point out that the neural language
model has a limitation in reproducing long-range correlation, another
statistical property of natural language. This understanding could provide a
direction for improving the architectures of neural networks.Comment: 21 pages, 11 figure
Nanolevel Surface Processing of Fine Particles by Waterjet Cavitation and Multifunction Cavitation to Improve the Photocatalytic Properties of Titanium Oxide
Titanium oxide particles were treated by water jet cavitation (WJC) generated and multifunction cavitation (MFC) using an ejector nozzle. Generation, growth, and collapse of cavitation are repeated with the particles of titanium oxide and platinum. Because the cavitation has an extremely high collapse pressure, the surface of the titanium oxide particles is processed by the microjets of cavitation in a reactor comprising the ejector nozzle. In the multifunction cavitation, ultrasonic irradiation of a waterjet during floating cavitation was used to generate microjets with hot spots. Hot working can be performed at the nanoscale on a material surface using this MFC process, resulting in morphological changes and variations in the surface electrochemical characteristics. The fundamental characteristics of multifunction cavitation were investigated theoretically and experimentally. Furthermore, the additional nozzle was put on the ejector nozzle in order to increase the temperature and pressure of bubble and the mechanism was clarified. The quantities of hydrogen and oxygen generated from titanium dioxide particles treated by multifunction cavitation in response to UV and visible light irradiation were remarkably increased compared to the amounts produced by particles treated by WJC processing. In this chapter, the methods and their results of processing particles by cavitation are introduced
Co-Training Realized Volatility Prediction Model with Neural Distributional Transformation
This paper shows a novel machine learning model for realized volatility (RV)
prediction using a normalizing flow, an invertible neural network. Since RV is
known to be skewed and have a fat tail, previous methods transform RV into
values that follow a latent distribution with an explicit shape and then apply
a prediction model. However, knowing that shape is non-trivial, and the
transformation result influences the prediction model. This paper proposes to
jointly train the transformation and the prediction model. The training process
follows a maximum-likelihood objective function that is derived from the
assumption that the prediction residuals on the transformed RV time series are
homogeneously Gaussian. The objective function is further approximated using an
expectation-maximum algorithm. On a dataset of 100 stocks, our method
significantly outperforms other methods using analytical or naive
neural-network transformations.Comment: Accepted at ICAIF'2
- …