Universal entropy of word ordering across linguistic families

Background The language faculty is probably the most distinctive feature of our species, and endows us with a unique ability to exchange highly structured information. In written language, information is encoded by the concatenation of basic symbols under grammatical and semantic constraints. As is also the case in other natural information carriers, the resulting symbolic sequences show a delicate balance between order and disorder. That balance is determined by the interplay between the diversity of symbols and by their specific ordering in the sequences. Here we used entropy to quantify the contribution of different organizational levels to the overall statistical structure of language. Methodology/Principal Findings We computed a relative entropy measure to quantify the degree of ordering in word sequences from languages belonging to several linguistic families. While a direct estimation of the overall entropy of language yielded values that varied for the different families considered, the relative entropy quantifying word ordering presented an almost constant value for all those families. Conclusions/Significance Our results indicate that despite the differences in the structure and vocabulary of the languages analyzed, the impact of word ordering in the structure of language is a statistical linguistic universal

Morphological complexity as a parameter of linguistic typology: Hungarian as a contact language

The paper builds on studies on Hungarian spoken outside Hungary (Fenyvesi (ed.) 2005), which show a change from synthetic to analytic expression in Hungarian in contact. It argues that a parameter of morphological complexity is helpful to account for most morphological changes. With one exception the changes follow the strategy of replicating use patterns (Heine & Kuteva 2005). Other changes arise by implication of a different typological system adopted by the new varieties of Hungarian (De Groot 2005a). A detailed comparison between Hungarian inside and outside Hungary in terms of linguistic complexity (Dahl 2004) confirm to the idea that languages in contact become linguistically more complex. The paper furthermore discusses the interaction between typology, language change by contact, and complexity

Natural Language Complexity and Machine Learning

International audienc