Regular Language Distance and Entropy

This paper addresses the problem of determining the distance between two regular languages. It will show how to expand Jaccard distance, which works on finite sets, to potentially-infinite regular languages. The entropy of a regular language plays a large role in the extension. Much of the paper is spent investigating the entropy of a regular language. This includes addressing issues that have required previous authors to rely on the upper limit of Shannon\u27s traditional formulation of channel capacity, because its limit does not always exist. The paper also includes proposing a new limit based formulation for the entropy of a regular language and proves that formulation to both exist and be equivalent to Shannon\u27s original formulation (when it exists). Additionally, the proposed formulation is shown to equal an analogous but formally quite different notion of topological entropy from Symbolic Dynamics -- consequently also showing Shannon\u27s original formulation to be equivalent to topological entropy. Surprisingly, the natural Jaccard-like entropy distance is trivial in most cases. Instead, the entropy sum distance metric is suggested, and shown to be granular in certain situations

Asymptotic Entropy of Random Walks on Regular Languages over a Finite Alphabet

We prove existence of asymptotic entropy of random walks on regular languages over a finite alphabet and we give formulas for it. Furthermore, we show that the entropy varies real-analytically in terms of probability measures of constant support, which describe the random walk. This setting applies, in particular, to random walks on virtually free groups.Comment: 47 pages, 2 figure

Complexity of Two-Dimensional Patterns

In dynamical systems such as cellular automata and iterated maps, it is often useful to look at a language or set of symbol sequences produced by the system. There are well-established classification schemes, such as the Chomsky hierarchy, with which we can measure the complexity of these sets of sequences, and thus the complexity of the systems which produce them. In this paper, we look at the first few levels of a hierarchy of complexity for two-or-more-dimensional patterns. We show that several definitions of ``regular language'' or ``local rule'' that are equivalent in d=1 lead to distinct classes in d >= 2. We explore the closure properties and computational complexity of these classes, including undecidability and L-, NL- and NP-completeness results. We apply these classes to cellular automata, in particular to their sets of fixed and periodic points, finite-time images, and limit sets. We show that it is undecidable whether a CA in d >= 2 has a periodic point of a given period, and that certain ``local lattice languages'' are not finite-time images or limit sets of any CA. We also show that the entropy of a d-dimensional CA's finite-time image cannot decrease faster than t^{-d} unless it maps every initial condition to a single homogeneous state.Comment: To appear in J. Stat. Phy

A broad-coverage distributed connectionist model of visual word recognition

In this study we describe a distributed connectionist model of morphological processing, covering a realistically sized sample of the English language. The purpose of this model is to explore how effects of discrete, hierarchically structured morphological paradigms, can arise as a result of the statistical sub-regularities in the mapping between word forms and word meanings. We present a model that learns to produce at its output a realistic semantic representation of a word, on presentation of a distributed representation of its orthography. After training, in three experiments, we compare the outputs of the model with the lexical decision latencies for large sets of English nouns and verbs. We show that the model has developed detailed representations of morphological structure, giving rise to effects analogous to those observed in visual lexical decision experiments. In addition, we show how the association between word form and word meaning also give rise to recently reported differences between regular and irregular verbs, even in their completely regular present-tense forms. We interpret these results as underlining the key importance for lexical processing of the statistical regularities in the mappings between form and meaning

Computation of distances for regular and context-free probabilistic languages

Several mathematical distances between probabilistic languages have been investigated in the literature, motivated by applications in language modeling, computational biology, syntactic pattern matching and machine learning. In most cases, only pairs of probabilistic regular languages were considered. In this paper we extend the previous results to pairs of languages generated by a probabilistic context-free grammar and a probabilistic finite automaton.PostprintPeer reviewe

Optimal lower bounds for quantum automata and random access codes

Consider the finite regular language L_n = {w0 : w \in {0,1}^*, |w| \le n}. It was shown by Ambainis, Nayak, Ta-Shma and Vazirani that while this language is accepted by a deterministic finite automaton of size O(n), any one-way quantum finite automaton (QFA) for it has size 2^{Omega(n/log n)}. This was based on the fact that the evolution of a QFA is required to be reversible. When arbitrary intermediate measurements are allowed, this intuition breaks down. Nonetheless, we show a 2^{Omega(n)} lower bound for such QFA for L_n, thus also improving the previous bound. The improved bound is obtained by simple entropy arguments based on Holevo's theorem. This method also allows us to obtain an asymptotically optimal (1-H(p))n bound for the dense quantum codes (random access codes) introduced by Ambainis et al. We then turn to Holevo's theorem, and show that in typical situations, it may be replaced by a tighter and more transparent in-probability bound.Comment: 8 pages, 1 figure, Latex2e. Extensive modifications have been made to increase clarity. To appear in FOCS'9

Minimally Constrained Stable Switched Systems and Application to Co-simulation

We propose an algorithm to restrict the switching signals of a constrained switched system in order to guarantee its stability, while at the same time attempting to keep the largest possible set of allowed switching signals. Our work is motivated by applications to (co-)simulation, where numerical stability is a hard constraint, but should be attained by restricting as little as possible the allowed behaviours of the simulators. We apply our results to certify the stability of an adaptive co-simulation orchestration algorithm, which selects the optimal switching signal at run-time, as a function of (varying) performance and accuracy requirements.Comment: Technical report complementing the following conference publication: Gomes, Cl\'audio, Beno\^it Legat, Rapha\"el Jungers, and Hans Vangheluwe. "Minimally Constrained Stable Switched Systems and Application to Co-Simulation." In IEEE Conference on Decision and Control. Miami Beach, FL, USA, 201