7 research outputs found
Scaling Laws in Human Language
Zipf's law on word frequency is observed in English, French, Spanish,
Italian, and so on, yet it does not hold for Chinese, Japanese or Korean
characters. A model for writing process is proposed to explain the above
difference, which takes into account the effects of finite vocabulary size.
Experiments, simulations and analytical solution agree well with each other.
The results show that the frequency distribution follows a power law with
exponent being equal to 1, at which the corresponding Zipf's exponent diverges.
Actually, the distribution obeys exponential form in the Zipf's plot. Deviating
from the Heaps' law, the number of distinct words grows with the text length in
three stages: It grows linearly in the beginning, then turns to a logarithmical
form, and eventually saturates. This work refines previous understanding about
Zipf's law and Heaps' law in language systems.Comment: 6 pages, 4 figure