Understanding scaling through history-dependent processes with collapsing sample space

History-dependent processes are ubiquitous in natural and social systems. Many such stochastic processes, especially those that are associated with complex systems, become more constrained as they unfold, meaning that their sample-space, or their set of possible outcomes, reduces as they age. We demonstrate that these sample-space reducing (SSR) processes necessarily lead to Zipf's law in the rank distributions of their outcomes. We show that by adding noise to SSR processes the corresponding rank distributions remain exact power-laws, $p(x)\sim x^{-\lambda}$, where the exponent directly corresponds to the mixing ratio of the SSR process and noise. This allows us to give a precise meaning to the scaling exponent in terms of the degree to how much a given process reduces its sample-space as it unfolds. Noisy SSR processes further allow us to explain a wide range of scaling exponents in frequency distributions ranging from $\alpha = 2$ to $\infty$. We discuss several applications showing how SSR processes can be used to understand Zipf's law in word frequencies, and how they are related to diffusion processes in directed networks, or ageing processes such as in fragmentation processes. SSR processes provide a new alternative to understand the origin of scaling in complex systems without the recourse to multiplicative, preferential, or self-organised critical processes.Comment: 7 pages, 5 figures in Proceedings of the National Academy of Sciences USA (published ahead of print April 13, 2015

Active causation and the origin of meaning

Purpose and meaning are necessary concepts for understanding mind and culture, but appear to be absent from the physical world and are not part of the explanatory framework of the natural sciences. Understanding how meaning (in the broad sense of the term) could arise from a physical world has proven to be a tough problem. The basic scheme of Darwinian evolution produces adaptations that only represent apparent ("as if") goals and meaning. Here I use evolutionary models to show that a slight, evolvable extension of the basic scheme is sufficient to produce genuine goals. The extension, targeted modulation of mutation rate, is known to be generally present in biological cells, and gives rise to two phenomena that are absent from the non-living world: intrinsic meaning and the ability to initiate goal-directed chains of causation (active causation). The extended scheme accomplishes this by utilizing randomness modulated by a feedback loop that is itself regulated by evolutionary pressure. The mechanism can be extended to behavioural variability as well, and thus shows how freedom of behaviour is possible. A further extension to communication suggests that the active exchange of intrinsic meaning between organisms may be the origin of consciousness, which in combination with active causation can provide a physical basis for the phenomenon of free will.Comment: revised and extende

Stochastic model for the vocabulary growth in natural languages

We propose a stochastic model for the number of different words in a given database which incorporates the dependence on the database size and historical changes. The main feature of our model is the existence of two different classes of words: (i) a finite number of core-words which have higher frequency and do not affect the probability of a new word to be used; and (ii) the remaining virtually infinite number of noncore-words which have lower frequency and once used reduce the probability of a new word to be used in the future. Our model relies on a careful analysis of the google-ngram database of books published in the last centuries and its main consequence is the generalization of Zipf's and Heaps' law to two scaling regimes. We confirm that these generalizations yield the best simple description of the data among generic descriptive models and that the two free parameters depend only on the language but not on the database. From the point of view of our model the main change on historical time scales is the composition of the specific words included in the finite list of core-words, which we observe to decay exponentially in time with a rate of approximately 30 words per year for English.Comment: corrected typos and errors in reference list; 10 pages text, 15 pages supplemental material; to appear in Physical Review

Implicit Simulations using Messaging Protocols

A novel algorithm for performing parallel, distributed computer simulations on the Internet using IP control messages is introduced. The algorithm employs carefully constructed ICMP packets which enable the required computations to be completed as part of the standard IP communication protocol. After providing a detailed description of the algorithm, experimental applications in the areas of stochastic neural networks and deterministic cellular automata are discussed. As an example of the algorithms potential power, a simulation of a deterministic cellular automaton involving 10^5 Internet connected devices was performed.Comment: 14 pages, 3 figure

A distributed procedure for computing stochastic expansions with Mathematica

The solution of a (stochastic) differential equation can be locally approximated by a (stochastic) expansion. If the vector field of the differential equation is a polynomial, the corresponding expansion is a linear combination of iterated integrals of the drivers and can be calculated using Picard Iterations. However, such expansions grow exponentially fast in their number of terms, due to their specific algebra, rendering their practical use limited. We present a Mathematica procedure that addresses this issue by reparametrizing the polynomials and distributing the load in as small as possible parts that can be processed and manipulated independently, thus alleviating large memory requirements and being perfectly suited for parallelized computation. We also present an iterative implementation of the shuffle product (as opposed to a recursive one, more usually implemented) as well as a fast way for calculating the expectation of iterated Stratonovich integrals for Brownian motion

Critical behavior in a cross-situational lexicon learning scenario

The associationist account for early word-learning is based on the co-occurrence between objects and words. Here we examine the performance of a simple associative learning algorithm for acquiring the referents of words in a cross-situational scenario affected by noise produced by out-of-context words. We find a critical value of the noise parameter $\gamma_c$ above which learning is impossible. We use finite-size scaling to show that the sharpness of the transition persists across a region of order $\tau^{-1/2}$ about $\gamma_c$, where $\tau$ is the number of learning trials, as well as to obtain the learning error (scaling function) in the critical region. In addition, we show that the distribution of durations of periods when the learning error is zero is a power law with exponent -3/2 at the critical point