2,649 research outputs found
The Computational Structure of Spike Trains
Neurons perform computations, and convey the results of those computations
through the statistical structure of their output spike trains. Here we present
a practical method, grounded in the information-theoretic analysis of
prediction, for inferring a minimal representation of that structure and for
characterizing its complexity. Starting from spike trains, our approach finds
their causal state models (CSMs), the minimal hidden Markov models or
stochastic automata capable of generating statistically identical time series.
We then use these CSMs to objectively quantify both the generalizable structure
and the idiosyncratic randomness of the spike train. Specifically, we show that
the expected algorithmic information content (the information needed to
describe the spike train exactly) can be split into three parts describing (1)
the time-invariant structure (complexity) of the minimal spike-generating
process, which describes the spike train statistically; (2) the randomness
(internal entropy rate) of the minimal spike-generating process; and (3) a
residual pure noise term not described by the minimal spike-generating process.
We use CSMs to approximate each of these quantities. The CSMs are inferred
nonparametrically from the data, making only mild regularity assumptions, via
the causal state splitting reconstruction algorithm. The methods presented here
complement more traditional spike train analyses by describing not only spiking
probability and spike train entropy, but also the complexity of a spike train's
structure. We demonstrate our approach using both simulated spike trains and
experimental data recorded in rat barrel cortex during vibrissa stimulation.Comment: Somewhat different format from journal version but same conten
An Analysis of Resting-State Functional Transcranial Doppler Recordings from Middle Cerebral Arteries
Functional transcrannial Doppler (fTCD) is used for monitoring the hemodynamics characteristics of major cerebral arteries. Its resting-state characteristics are known only when considering the maximal velocity corresponding to the highest Doppler shift (so called the envelope signals). Significantly more information about the resting-state fTCD can be gained when considering the raw cerebral blood flow velocity (CBFV) recordings. In this paper, we considered simultaneously acquired envelope and raw CBFV signals. Specifically, we collected bilateral CBFV recordings from left and right middle cerebral arteries using 20 healthy subjects (10 females). The data collection lasted for 15 minutes. The subjects were asked to remain awake, stay silent, and try to remain thought-free during the data collection. Time, frequency and time-frequency features were extracted from both the raw and the envelope CBFV signals. The effects of age, sex and body-mass index were examined on the extracted features. The results showed that the raw CBFV signals had a higher frequency content, and its temporal structures were almost uncorrelated. The information-theoretic features showed that the raw recordings from left and right middle cerebral arteries had higher content of mutual information than the envelope signals. Age and body-mass index did not have statistically significant effects on the extracted features. Sex-based differences were observed in all three domains and for both, the envelope signals and the raw CBFV signals. These findings indicate that the raw CBFV signals provide valuable information about the cerebral blood flow which can be utilized in further validation of fTCD as a clinical tool. © 2013 Sejdić et al
Computational Mechanics of Input-Output Processes: Structured transformations and the -transducer
Computational mechanics quantifies structure in a stochastic process via its
causal states, leading to the process's minimal, optimal predictor---the
-machine. We extend computational mechanics to communication channels
between two processes, obtaining an analogous optimal model---the
-transducer---of the stochastic mapping between them. Here, we lay
the foundation of a structural analysis of communication channels, treating
joint processes and processes with input. The result is a principled structural
analysis of mechanisms that support information flow between processes. It is
the first in a series on the structural information theory of memoryful
channels, channel composition, and allied conditional information measures.Comment: 30 pages, 19 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/et1.htm; Updated to conform to
published version plus additional corrections and update
Local information transfer as a spatiotemporal filter for complex systems
We present a measure of local information transfer, derived from an existing
averaged information-theoretical measure, namely transfer entropy. Local
transfer entropy is used to produce profiles of the information transfer into
each spatiotemporal point in a complex system. These spatiotemporal profiles
are useful not only as an analytical tool, but also allow explicit
investigation of different parameter settings and forms of the transfer entropy
metric itself. As an example, local transfer entropy is applied to cellular
automata, where it is demonstrated to be a novel method of filtering for
coherent structure. More importantly, local transfer entropy provides the first
quantitative evidence for the long-held conjecture that the emergent traveling
coherent structures known as particles (both gliders and domain walls, which
have analogues in many physical processes) are the dominant information
transfer agents in cellular automata.Comment: 12 page
Inducing Probabilistic Grammars by Bayesian Model Merging
We describe a framework for inducing probabilistic grammars from corpora of
positive samples. First, samples are {\em incorporated} by adding ad-hoc rules
to a working grammar; subsequently, elements of the model (such as states or
nonterminals) are {\em merged} to achieve generalization and a more compact
representation. The choice of what to merge and when to stop is governed by the
Bayesian posterior probability of the grammar given the data, which formalizes
a trade-off between a close fit to the data and a default preference for
simpler models (`Occam's Razor'). The general scheme is illustrated using three
types of probabilistic grammars: Hidden Markov models, class-based -grams,
and stochastic context-free grammars.Comment: To appear in Grammatical Inference and Applications, Second
International Colloquium on Grammatical Inference; Springer Verlag, 1994. 13
page
An Algorithm for Pattern Discovery in Time Series
We present a new algorithm for discovering patterns in time series and other
sequential data. We exhibit a reliable procedure for building the minimal set
of hidden, Markovian states that is statistically capable of producing the
behavior exhibited in the data -- the underlying process's causal states.
Unlike conventional methods for fitting hidden Markov models (HMMs) to data,
our algorithm makes no assumptions about the process's causal architecture (the
number of hidden states and their transition structure), but rather infers it
from the data. It starts with assumptions of minimal structure and introduces
complexity only when the data demand it. Moreover, the causal states it infers
have important predictive optimality properties that conventional HMM states
lack. We introduce the algorithm, review the theory behind it, prove its
asymptotic reliability, use large deviation theory to estimate its rate of
convergence, and compare it to other algorithms which also construct HMMs from
data. We also illustrate its behavior on an example process, and report
selected numerical results from an implementation.Comment: 26 pages, 5 figures; 5 tables;
http://www.santafe.edu/projects/CompMech Added discussion of algorithm
parameters; improved treatment of convergence and time complexity; added
comparison to older method
Statistical approaches for natural language modelling and monotone statistical machine translation
Esta tesis reune algunas contribuciones al reconocimiento de formas estadÃstico y, más especÃcamente, a varias tareas del procesamiento del lenguaje natural. Varias técnicas estadÃsticas bien conocidas se revisan en esta tesis, a saber: estimación paramétrica, diseño de la función de pérdida y modelado estadÃstico. Estas técnicas se aplican a varias tareas del procesamiento del lenguajes natural tales como clasicación de documentos, modelado del lenguaje natural
y traducción automática estadÃstica.
En relación con la estimación paramétrica, abordamos el problema del suavizado proponiendo una nueva técnica de estimación por máxima verosimilitud con dominio restringido (CDMLEa ). La técnica CDMLE evita la necesidad de la etapa de suavizado que propicia la pérdida de las propiedades del estimador máximo verosÃmil. Esta técnica se aplica a clasicación de documentos mediante el clasificador Naive Bayes. Más tarde, la técnica CDMLE se extiende a la estimación por máxima verosimilitud por leaving-one-out aplicandola al suavizado de modelos de lenguaje. Los resultados obtenidos en varias tareas de modelado del lenguaje natural, muestran una mejora en términos de perplejidad.
En a la función de pérdida, se estudia cuidadosamente el diseño de funciones de pérdida diferentes a la 0-1. El estudio se centra en aquellas funciones de pérdida que reteniendo una complejidad de decodificación similar a la función 0-1, proporcionan una mayor flexibilidad. Analizamos y presentamos varias funciones de pérdida en varias tareas de traducción automática y con varios modelos de traducción. También, analizamos algunas reglas de traducción que destacan por causas prácticas tales como la regla de traducción directa; y, asà mismo, profundizamos en la comprensión de los modelos log-lineares, que son de hecho, casos particulares de funciones de pérdida.
Finalmente, se proponen varios modelos de traducción monótonos basados en técnicas de modelado estadÃstico .Andrés Ferrer, J. (2010). Statistical approaches for natural language modelling and monotone statistical machine translation [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/7109Palanci
- …