58,045 research outputs found
An algorithm for the word entropy
For any infinite word on a finite alphabet , the complexity function
of is the sequence counting, for each non-negative , the number
of words of length on the alphabet that are factors of the
infinite word and the the entropy of is the quantity
. For any given function
with exponential growth, Mauduit and Moreira introduced in [MM17] the notion of
word entropy and
showed its links with fractal dimensions of sets of infinite sequences with
complexity function bounded by . The goal of this work is to give an
algorithm to estimate with arbitrary precision from finitely many
values of
Inducing Features of Random Fields
We present a technique for constructing random fields from a set of training
samples. The learning paradigm builds increasingly complex fields by allowing
potential functions, or features, that are supported by increasingly large
subgraphs. Each feature has a weight that is trained by minimizing the
Kullback-Leibler divergence between the model and the empirical distribution of
the training data. A greedy algorithm determines how features are incrementally
added to the field and an iterative scaling algorithm is used to estimate the
optimal values of the weights.
The statistical modeling techniques introduced in this paper differ from
those common to much of the natural language processing literature since there
is no probabilistic finite state or push-down automaton on which the model is
built. Our approach also differs from the techniques common to the computer
vision literature in that the underlying random fields are non-Markovian and
have a large number of parameters that must be estimated. Relations to other
learning approaches including decision trees and Boltzmann machines are given.
As a demonstration of the method, we describe its application to the problem of
automatic word classification in natural language processing.
Key words: random field, Kullback-Leibler divergence, iterative scaling,
divergence geometry, maximum entropy, EM algorithm, statistical learning,
clustering, word morphology, natural language processingComment: 34 pages, compressed postscrip
Many Roads to Synchrony: Natural Time Scales and Their Algorithms
We consider two important time scales---the Markov and cryptic orders---that
monitor how an observer synchronizes to a finitary stochastic process. We show
how to compute these orders exactly and that they are most efficiently
calculated from the epsilon-machine, a process's minimal unifilar model.
Surprisingly, though the Markov order is a basic concept from stochastic
process theory, it is not a probabilistic property of a process. Rather, it is
a topological property and, moreover, it is not computable from any
finite-state model other than the epsilon-machine. Via an exhaustive survey, we
close by demonstrating that infinite Markov and infinite cryptic orders are a
dominant feature in the space of finite-memory processes. We draw out the roles
played in statistical mechanical spin systems by these two complementary length
scales.Comment: 17 pages, 16 figures:
http://cse.ucdavis.edu/~cmg/compmech/pubs/kro.htm. Santa Fe Institute Working
Paper 10-11-02
Minimally Constrained Stable Switched Systems and Application to Co-simulation
We propose an algorithm to restrict the switching signals of a constrained
switched system in order to guarantee its stability, while at the same time
attempting to keep the largest possible set of allowed switching signals. Our
work is motivated by applications to (co-)simulation, where numerical stability
is a hard constraint, but should be attained by restricting as little as
possible the allowed behaviours of the simulators. We apply our results to
certify the stability of an adaptive co-simulation orchestration algorithm,
which selects the optimal switching signal at run-time, as a function of
(varying) performance and accuracy requirements.Comment: Technical report complementing the following conference publication:
Gomes, Cl\'audio, Beno\^it Legat, Rapha\"el Jungers, and Hans Vangheluwe.
"Minimally Constrained Stable Switched Systems and Application to
Co-Simulation." In IEEE Conference on Decision and Control. Miami Beach, FL,
USA, 201
Discriminated Belief Propagation
Near optimal decoding of good error control codes is generally a difficult
task. However, for a certain type of (sufficiently) good codes an efficient
decoding algorithm with near optimal performance exists. These codes are
defined via a combination of constituent codes with low complexity trellis
representations. Their decoding algorithm is an instance of (loopy) belief
propagation and is based on an iterative transfer of constituent beliefs. The
beliefs are thereby given by the symbol probabilities computed in the
constituent trellises. Even though weak constituent codes are employed close to
optimal performance is obtained, i.e., the encoder/decoder pair (almost)
achieves the information theoretic capacity. However, (loopy) belief
propagation only performs well for a rather specific set of codes, which limits
its applicability.
In this paper a generalisation of iterative decoding is presented. It is
proposed to transfer more values than just the constituent beliefs. This is
achieved by the transfer of beliefs obtained by independently investigating
parts of the code space. This leads to the concept of discriminators, which are
used to improve the decoder resolution within certain areas and defines
discriminated symbol beliefs. It is shown that these beliefs approximate the
overall symbol probabilities. This leads to an iteration rule that (below
channel capacity) typically only admits the solution of the overall decoding
problem. Via a Gauss approximation a low complexity version of this algorithm
is derived. Moreover, the approach may then be applied to a wide range of
channel maps without significant complexity increase
- …