Diffusion of Context and Credit Information in Markovian Models

This paper studies the problem of ergodicity of transition probability matrices in Markovian models, such as hidden Markov models (HMMs), and how it makes very difficult the task of learning to represent long-term context for sequential data. This phenomenon hurts the forward propagation of long-term context information, as well as learning a hidden state representation to represent long-term context, which depends on propagating credit information backwards in time. Using results from Markov chain theory, we show that this problem of diffusion of context and credit is reduced when the transition probabilities approach 0 or 1, i.e., the transition probability matrices are sparse and the model essentially deterministic. The results found in this paper apply to learning approaches based on continuous optimization, such as gradient descent and the Baum-Welch algorithm.Comment: See http://www.jair.org/ for any accompanying file

Symmetry Decomposition of Chaotic Dynamics

Discrete symmetries of dynamical flows give rise to relations between periodic orbits, reduce the dynamics to a fundamental domain, and lead to factorizations of zeta functions. These factorizations in turn reduce the labor and improve the convergence of cycle expansions for classical and quantum spectra associated with the flow. In this paper the general formalism is developed, with the $N$-disk pinball model used as a concrete example and a series of physically interesting cases worked out in detail.Comment: CYCLER Paper 93mar01

Subclasses of Presburger Arithmetic and the Weak EXP Hierarchy

It is shown that for any fixed $i>0$, the $\Sigma_{i+1}$-fragment of Presburger arithmetic, i.e., its restriction to $i+1$ quantifier alternations beginning with an existential quantifier, is complete for $\mathsf{\Sigma}^{\mathsf{EXP}}_{i}$, the $i$-th level of the weak EXP hierarchy, an analogue to the polynomial-time hierarchy residing between $\mathsf{NEXP}$ and $\mathsf{EXPSPACE}$. This result completes the computational complexity landscape for Presburger arithmetic, a line of research which dates back to the seminal work by Fischer & Rabin in 1974. Moreover, we apply some of the techniques developed in the proof of the lower bound in order to establish bounds on sets of naturals definable in the $\Sigma_1$-fragment of Presburger arithmetic: given a $\Sigma_1$-formula $\Phi(x)$, it is shown that the set of non-negative solutions is an ultimately periodic set whose period is at most doubly-exponential and that this bound is tight.Comment: 10 pages, 2 figure

Exploiting lattice structures in shape grammar implementations

The ability to work with ambiguity and compute new designs based on both defined and emergent shapes are unique advantages of shape grammars. Realizing these benefits in design practice requires the implementation of general purpose shape grammar interpreters that support: (a) the detection of arbitrary subshapes in arbitrary shapes and (b) the application of shape rules that use these subshapes to create new shapes. The complexity of currently available interpreters results from their combination of shape computation (for subshape detection and the application of rules) with computational geometry (for the geometric operations need to generate new shapes). This paper proposes a shape grammar implementation method for three-dimensional circular arcs represented as rational quadratic Bézier curves based on lattice theory that reduces this complexity by separating steps in a shape computation process from the geometrical operations associated with specific grammars and shapes. The method is demonstrated through application to two well-known shape grammars: Stiny's triangles grammar and Jowers and Earl's trefoil grammar. A prototype computer implementation of an interpreter kernel has been built and its application to both grammars is presented. The use of Bézier curves in three dimensions opens the possibility to extend shape grammar implementations to cover the wider range of applications that are needed before practical implementations for use in real life product design and development processes become feasible

Gibbs and Quantum Discrete Spaces

Gibbs measure is one of the central objects of the modern probability, mathematical statistical physics and euclidean quantum field theory. Here we define and study its natural generalization for the case when the space, where the random field is defined is itself random. Moreover, this randomness is not given apriori and independently of the configuration, but rather they depend on each other, and both are given by Gibbs procedure; We call the resulting object a Gibbs family because it parametrizes Gibbs fields on different graphs in the support of the distribution. We study also quantum (KMS) analog of Gibbs families. Various applications to discrete quantum gravity are given.Comment: 37 pages, 2 figure

Advances and applications of automata on words and trees : abstracts collection

From 12.12.2010 to 17.12.2010, the Dagstuhl Seminar 10501 "Advances and Applications of Automata on Words and Trees" was held in Schloss Dagstuhl - Leibniz Center for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Calibrating Generative Models: The Probabilistic Chomsky-Schützenberger Hierarchy

A probabilistic Chomsky–Schützenberger hierarchy of grammars is introduced and studied, with the aim of understanding the expressive power of generative models. We offer characterizations of the distributions definable at each level of the hierarchy, including probabilistic regular, context-free, (linear) indexed, context-sensitive, and unrestricted grammars, each corresponding to familiar probabilistic machine classes. Special attention is given to distributions on (unary notations for) positive integers. Unlike in the classical case where the "semi-linear" languages all collapse into the regular languages, using analytic tools adapted from the classical setting we show there is no collapse in the probabilistic hierarchy: more distributions become definable at each level. We also address related issues such as closure under probabilistic conditioning