Inducing Language Networks from Continuous Space Word Representations

Recent advancements in unsupervised feature learning have developed powerful latent representations of words. However, it is still not clear what makes one representation better than another and how we can learn the ideal representation. Understanding the structure of latent spaces attained is key to any future advancement in unsupervised learning. In this work, we introduce a new view of continuous space word representations as language networks. We explore two techniques to create language networks from learned features by inducing them for two popular word representation methods and examining the properties of their resulting networks. We find that the induced networks differ from other methods of creating language networks, and that they contain meaningful community structure.Comment: 14 page

Construction of equilibrium networks with an energy function

We construct equilibrium networks by introducing an energy function depending on the degree of each node as well as the product of neighboring degrees. With this topological energy function, networks constitute a canonical ensemble, which follows the Boltzmann distribution for given temperature. It is observed that the system undergoes a topological phase transition from a random network to a star or a fully-connected network as the temperature is lowered. Both mean-field analysis and numerical simulations reveal strong first-order phase transitions at temperatures which decrease logarithmically with the system size. Quantitative discrepancies of the simulation results from the mean-field prediction are discussed in view of the strong first-order nature.Comment: To appear in J. Phys.

Zipf's Law and Avoidance of Excessive Synonymy

Zipf's law states that if words of language are ranked in the order of decreasing frequency in texts, the frequency of a word is inversely proportional to its rank. It is very robust as an experimental observation, but to date it escaped satisfactory theoretical explanation. We suggest that Zipf's law may arise from the evolution of word semantics dominated by expansion of meanings and competition of synonyms.Comment: 47 pages; fixed reference list missing in v.

Statistical Laws Governing Fluctuations in Word Use from Word Birth to Word Death

We analyze the dynamic properties of 10^7 words recorded in English, Spanish and Hebrew over the period 1800--2008 in order to gain insight into the coevolution of language and culture. We report language independent patterns useful as benchmarks for theoretical models of language evolution. A significantly decreasing (increasing) trend in the birth (death) rate of words indicates a recent shift in the selection laws governing word use. For new words, we observe a peak in the growth-rate fluctuations around 40 years after introduction, consistent with the typical entry time into standard dictionaries and the human generational timescale. Pronounced changes in the dynamics of language during periods of war shows that word correlations, occurring across time and between words, are largely influenced by coevolutionary social, technological, and political factors. We quantify cultural memory by analyzing the long-term correlations in the use of individual words using detrended fluctuation analysis.Comment: Version 1: 31 pages, 17 figures, 3 tables. Version 2 is streamlined, eliminates substantial material and incorporates referee comments: 19 pages, 14 figures, 3 table

Scaling Laws in Human Language

Zipf's law on word frequency is observed in English, French, Spanish, Italian, and so on, yet it does not hold for Chinese, Japanese or Korean characters. A model for writing process is proposed to explain the above difference, which takes into account the effects of finite vocabulary size. Experiments, simulations and analytical solution agree well with each other. The results show that the frequency distribution follows a power law with exponent being equal to 1, at which the corresponding Zipf's exponent diverges. Actually, the distribution obeys exponential form in the Zipf's plot. Deviating from the Heaps' law, the number of distinct words grows with the text length in three stages: It grows linearly in the beginning, then turns to a logarithmical form, and eventually saturates. This work refines previous understanding about Zipf's law and Heaps' law in language systems.Comment: 6 pages, 4 figure

On the Complex Network Structure of Musical Pieces: Analysis of Some Use Cases from Different Music Genres

This paper focuses on the modeling of musical melodies as networks. Notes of a melody can be treated as nodes of a network. Connections are created whenever notes are played in sequence. We analyze some main tracks coming from different music genres, with melodies played using different musical instruments. We find out that the considered networks are, in general, scale free networks and exhibit the small world property. We measure the main metrics and assess whether these networks can be considered as formed by sub-communities. Outcomes confirm that peculiar features of the tracks can be extracted from this analysis methodology. This approach can have an impact in several multimedia applications such as music didactics, multimedia entertainment, and digital music generation.Comment: accepted to Multimedia Tools and Applications, Springe

Point-occurrence self-similarity in crackling-noise systems and in other complex systems

It has been recently found that a number of systems displaying crackling noise also show a remarkable behavior regarding the temporal occurrence of successive events versus their size: a scaling law for the probability distributions of waiting times as a function of a minimum size is fulfilled, signaling the existence on those systems of self-similarity in time-size. This property is also present in some non-crackling systems. Here, the uncommon character of the scaling law is illustrated with simple marked renewal processes, built by definition with no correlations. Whereas processes with a finite mean waiting time do not fulfill a scaling law in general and tend towards a Poisson process in the limit of very high sizes, processes without a finite mean tend to another class of distributions, characterized by double power-law waiting-time densities. This is somehow reminiscent of the generalized central limit theorem. A model with short-range correlations is not able to escape from the attraction of those limit distributions. A discussion on open problems in the modeling of these properties is provided.Comment: Submitted to J. Stat. Mech. for the proceedings of UPON 2008 (Lyon), topic: crackling nois

Conferred resistance to Botrytis cinerea in Lilium by overexpression of the RCH10 chitinase gene

The production of ornamentals is an important global industry, with Lilium being one of the six major bulb crops in the world. The international trade in ornamentals is in the order of £60-75 billion and is expected to increase worldwide by 2-4 % per annum. The continued success of the floriculture industry depends on the introduction of new species/cultivars with major alterations in key agronomic characteristics, such as resistance to pathogens. Fungal diseases are the cause of reduced yields and marketable quality of cultivated plants, including ornamental species. The fungal pathogen Botrytis causes extreme economic losses to a wide range of crop species, including ornamentals such as Lilium. Agrobacterium-mediated transformation was used to develop Lilium oriental cv. ‘Star Gazer’ plants that ectopically overexpress the Rice Chitinase 10 gene (RCH10), under control of the CaMV35S promoter. Levels of conferred resistance linked to chitinase expression were evaluated by infection with Botrytis cinerea; sporulation was reduced in an in vitro assay and the relative expression of the RCH10 gene was determined by quantitative Reverse-Transcriptase PCR. The extent of resistance to Botrytis, compared to that of the wild type plants, showed a direct correlation with the level of chitinase gene expression. Transgenic plants grown to flowering showed no detrimental phenotypic effects associated with transgene expression. This is the first report of Lilium plants with resistance to Botrytis cinerea generated by a transgenic approach