173 research outputs found
Long-Range Correlation Underlying Childhood Language and Generative Models
Long-range correlation, a property of time series exhibiting long-term
memory, is mainly studied in the statistical physics domain and has been
reported to exist in natural language. Using a state-of-the-art method for such
analysis, long-range correlation is first shown to occur in long CHILDES data
sets. To understand why, Bayesian generative models of language, originally
proposed in the cognitive scientific domain, are investigated. Among
representative models, the Simon model was found to exhibit surprisingly good
long-range correlation, but not the Pitman-Yor model. Since the Simon model is
known not to correctly reflect the vocabulary growth of natural language, a
simple new model is devised as a conjunct of the Simon and Pitman-Yor models,
such that long-range correlation holds with a correct vocabulary growth rate.
The investigation overall suggests that uniform sampling is one cause of
long-range correlation and could thus have a relation with actual linguistic
processes
Do Neural Nets Learn Statistical Laws behind Natural Language?
The performance of deep learning in natural language processing has been
spectacular, but the reasons for this success remain unclear because of the
inherent complexity of deep learning. This paper provides empirical evidence of
its effectiveness and of a limitation of neural networks for language
engineering. Precisely, we demonstrate that a neural language model based on
long short-term memory (LSTM) effectively reproduces Zipf's law and Heaps' law,
two representative statistical properties underlying natural language. We
discuss the quality of reproducibility and the emergence of Zipf's law and
Heaps' law as training progresses. We also point out that the neural language
model has a limitation in reproducing long-range correlation, another
statistical property of natural language. This understanding could provide a
direction for improving the architectures of neural networks.Comment: 21 pages, 11 figure
Co-Training Realized Volatility Prediction Model with Neural Distributional Transformation
This paper shows a novel machine learning model for realized volatility (RV)
prediction using a normalizing flow, an invertible neural network. Since RV is
known to be skewed and have a fat tail, previous methods transform RV into
values that follow a latent distribution with an explicit shape and then apply
a prediction model. However, knowing that shape is non-trivial, and the
transformation result influences the prediction model. This paper proposes to
jointly train the transformation and the prediction model. The training process
follows a maximum-likelihood objective function that is derived from the
assumption that the prediction residuals on the transformed RV time series are
homogeneously Gaussian. The objective function is further approximated using an
expectation-maximum algorithm. On a dataset of 100 stocks, our method
significantly outperforms other methods using analytical or naive
neural-network transformations.Comment: Accepted at ICAIF'2
05382 Abstracts Collection -- Efficient Text Entry
From 21.09.05 to 24.09.05, the Dagstuhl Seminar 05382 ``Efficient Text Entry\u27\u27 was held in the International Conference and Research Center (IBFI),
Schloss Dagstuhl.
During the seminar, several participants presented their current
research, and ongoing work and open problems were discussed. Abstracts of
the presentations given during the seminar as well as abstracts of
seminar results and ideas are put together in this paper. The first section
describes the seminar topics and goals in general.
Links to extended abstracts or full papers are provided, if available
Long-term oxygen therapy in Japan: history, present status, and current problems
Historically, the progress of long term-oxygen therapy (LTOT) in Japan has been characterized by collaboration among academicgroups, policy makers, and industrial companies. The public health insurance program has covered the cost of LTOT since 1985.Thomas Petty’s group in Denver enthusiastically carried out the public implementation of LTOT and conveyed the concept of pulmonaryrehabilitation for the processing with LTOT. Although the target diseases of LTOT in Japan tended to be chronic obstructivepulmonary disease or sequelae of primary lung tuberculosis, it was soon applied for cardiac diseases as well as other pulmonarydiseases. Together with increasing medical costs for geriatric patients, the political conversion from hospital based care ofa traditional style to home care system has been performed, with two background reasons: the improvement of quality of life ofpatients and the reduction of the medical expense. Presently, LTOT plays a pivotal role in the successful implementation of homerespiratory care for elderly patients. In addition, this promotes comprehensive pulmonary rehabilitation, a team approach, andclose liaisons between primary care and hospitals. Currently, the total number of patients using LTOT exceeds 150,000. In Japan,LTOT resulted in an advancement in the medical care as well as in administrative decision to introduce it as a nationwide systemafter analyzing the results of opinion polls of patients with respiratory failure. However, the recent great earthquake in East Japanrevealed that many unresolved problems remain for these patients, and these issues are of great concern.Historically, the progress of long term-oxygen therapy (LTOT) in Japan has been characterized by collaboration among academicgroups, policy makers, and industrial companies. The public health insurance program has covered the cost of LTOT since 1985.Thomas Petty’s group in Denver enthusiastically carried out the public implementation of LTOT and conveyed the concept of pulmonaryrehabilitation for the processing with LTOT. Although the target diseases of LTOT in Japan tended to be chronic obstructivepulmonary disease or sequelae of primary lung tuberculosis, it was soon applied for cardiac diseases as well as other pulmonarydiseases. Together with increasing medical costs for geriatric patients, the political conversion from hospital based care ofa traditional style to home care system has been performed, with two background reasons: the improvement of quality of life ofpatients and the reduction of the medical expense. Presently, LTOT plays a pivotal role in the successful implementation of homerespiratory care for elderly patients. In addition, this promotes comprehensive pulmonary rehabilitation, a team approach, andclose liaisons between primary care and hospitals. Currently, the total number of patients using LTOT exceeds 150,000. In Japan,LTOT resulted in an advancement in the medical care as well as in administrative decision to introduce it as a nationwide systemafter analyzing the results of opinion polls of patients with respiratory failure. However, the recent great earthquake in East Japanrevealed that many unresolved problems remain for these patients, and these issues are of great concern
- …