11,113 research outputs found
An Algorithm for Pattern Discovery in Time Series
We present a new algorithm for discovering patterns in time series and other
sequential data. We exhibit a reliable procedure for building the minimal set
of hidden, Markovian states that is statistically capable of producing the
behavior exhibited in the data -- the underlying process's causal states.
Unlike conventional methods for fitting hidden Markov models (HMMs) to data,
our algorithm makes no assumptions about the process's causal architecture (the
number of hidden states and their transition structure), but rather infers it
from the data. It starts with assumptions of minimal structure and introduces
complexity only when the data demand it. Moreover, the causal states it infers
have important predictive optimality properties that conventional HMM states
lack. We introduce the algorithm, review the theory behind it, prove its
asymptotic reliability, use large deviation theory to estimate its rate of
convergence, and compare it to other algorithms which also construct HMMs from
data. We also illustrate its behavior on an example process, and report
selected numerical results from an implementation.Comment: 26 pages, 5 figures; 5 tables;
http://www.santafe.edu/projects/CompMech Added discussion of algorithm
parameters; improved treatment of convergence and time complexity; added
comparison to older method
The Computational Structure of Spike Trains
Neurons perform computations, and convey the results of those computations
through the statistical structure of their output spike trains. Here we present
a practical method, grounded in the information-theoretic analysis of
prediction, for inferring a minimal representation of that structure and for
characterizing its complexity. Starting from spike trains, our approach finds
their causal state models (CSMs), the minimal hidden Markov models or
stochastic automata capable of generating statistically identical time series.
We then use these CSMs to objectively quantify both the generalizable structure
and the idiosyncratic randomness of the spike train. Specifically, we show that
the expected algorithmic information content (the information needed to
describe the spike train exactly) can be split into three parts describing (1)
the time-invariant structure (complexity) of the minimal spike-generating
process, which describes the spike train statistically; (2) the randomness
(internal entropy rate) of the minimal spike-generating process; and (3) a
residual pure noise term not described by the minimal spike-generating process.
We use CSMs to approximate each of these quantities. The CSMs are inferred
nonparametrically from the data, making only mild regularity assumptions, via
the causal state splitting reconstruction algorithm. The methods presented here
complement more traditional spike train analyses by describing not only spiking
probability and spike train entropy, but also the complexity of a spike train's
structure. We demonstrate our approach using both simulated spike trains and
experimental data recorded in rat barrel cortex during vibrissa stimulation.Comment: Somewhat different format from journal version but same conten
Measuring information-transfer delays
In complex networks such as gene networks, traffic systems or brain circuits it is important to understand how long it takes for the different parts of the network to effectively influence one another. In the brain, for example, axonal delays between brain areas can amount to several tens of milliseconds, adding an intrinsic component to any timing-based processing of information. Inferring neural interaction delays is thus needed to interpret the information transfer revealed by any analysis of directed interactions across brain structures. However, a robust estimation of interaction delays from neural activity faces several challenges if modeling assumptions on interaction mechanisms are wrong or cannot be made. Here, we propose a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, which allows a model-free exploration of interactions. In particular, we extend transfer entropy to account for delayed source-target interactions, while crucially retaining the conditioning on the embedded target state at the immediately previous time step. We prove that this particular extension is indeed guaranteed to identify interaction delays between two coupled systems and is the only relevant option in keeping with Wiener’s principle of causality. We demonstrate the performance of our approach in detecting interaction delays on finite data by numerical simulations of stochastic and deterministic processes, as well as on local field potential recordings. We also show the ability of the extended transfer entropy to detect the presence of multiple delays, as well as feedback loops. While evaluated on neuroscience data, we expect the estimator to be useful in other fields dealing with network dynamics
Model-free reconstruction of neuronal network connectivity from calcium imaging signals
A systematic assessment of global neural network connectivity through direct
electrophysiological assays has remained technically unfeasible even in
dissociated neuronal cultures. We introduce an improved algorithmic approach
based on Transfer Entropy to reconstruct approximations to network structural
connectivities from network activity monitored through calcium fluorescence
imaging. Based on information theory, our method requires no prior assumptions
on the statistics of neuronal firing and neuronal connections. The performance
of our algorithm is benchmarked on surrogate time-series of calcium
fluorescence generated by the simulated dynamics of a network with known
ground-truth topology. We find that the effective network topology revealed by
Transfer Entropy depends qualitatively on the time-dependent dynamic state of
the network (e.g., bursting or non-bursting). We thus demonstrate how
conditioning with respect to the global mean activity improves the performance
of our method. [...] Compared to other reconstruction strategies such as
cross-correlation or Granger Causality methods, our method based on improved
Transfer Entropy is remarkably more accurate. In particular, it provides a good
reconstruction of the network clustering coefficient, allowing to discriminate
between weakly or strongly clustered topologies, whereas on the other hand an
approach based on cross-correlations would invariantly detect artificially high
levels of clustering. Finally, we present the applicability of our method to
real recordings of in vitro cortical cultures. We demonstrate that these
networks are characterized by an elevated level of clustering compared to a
random graph (although not extreme) and by a markedly non-local connectivity.Comment: 54 pages, 8 figures (+9 supplementary figures), 1 table; submitted
for publicatio
Markov processes follow from the principle of Maximum Caliber
Markov models are widely used to describe processes of stochastic dynamics.
Here, we show that Markov models are a natural consequence of the dynamical
principle of Maximum Caliber. First, we show that when there are different
possible dynamical trajectories in a time-homogeneous process, then the only
type of process that maximizes the path entropy, for any given singlet
statistics, is a sequence of identical, independently distributed (i.i.d.)
random variables, which is the simplest Markov process. If the data is in the
form of sequentially pairwise statistics, then maximizing the caliber dictates
that the process is Markovian with a uniform initial distribution. Furthermore,
if an initial non-uniform dynamical distribution is known, or multiple
trajectories are conditioned on an initial state, then the Markov process is
still the only one that maximizes the caliber. Second, given a model, MaxCal
can be used to compute the parameters of that model. We show that this
procedure is equivalent to the maximum-likelihood method of inference in the
theory of statistics.Comment: 4 page
- …