15,299 research outputs found
On universal estimates for binary renewal processes
A binary renewal process is a stochastic process taking values in
where the lengths of the runs of 1's between successive zeros are
independent. After observing one would like to predict the
future behavior, and the problem of universal estimators is to do so without
any prior knowledge of the distribution. We prove a variety of results of this
type, including universal estimates for the expected time to renewal as well as
estimates for the conditional distribution of the time to renewal. Some of our
results require a moment condition on the time to renewal and we show by an
explicit construction how some moment condition is necessary.Comment: Published in at http://dx.doi.org/10.1214/07-AAP512 the Annals of
Applied Probability (http://www.imstat.org/aap/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Time Resolution Dependence of Information Measures for Spiking Neurons: Atoms, Scaling, and Universality
The mutual information between stimulus and spike-train response is commonly
used to monitor neural coding efficiency, but neuronal computation broadly
conceived requires more refined and targeted information measures of
input-output joint processes. A first step towards that larger goal is to
develop information measures for individual output processes, including
information generation (entropy rate), stored information (statistical
complexity), predictable information (excess entropy), and active information
accumulation (bound information rate). We calculate these for spike trains
generated by a variety of noise-driven integrate-and-fire neurons as a function
of time resolution and for alternating renewal processes. We show that their
time-resolution dependence reveals coarse-grained structural properties of
interspike interval statistics; e.g., -entropy rates that diverge less
quickly than the firing rate indicate interspike interval correlations. We also
find evidence that the excess entropy and regularized statistical complexity of
different types of integrate-and-fire neurons are universal in the
continuous-time limit in the sense that they do not depend on mechanism
details. This suggests a surprising simplicity in the spike trains generated by
these model neurons. Interestingly, neurons with gamma-distributed ISIs and
neurons whose spike trains are alternating renewal processes do not fall into
the same universality class. These results lead to two conclusions. First, the
dependence of information measures on time resolution reveals mechanistic
details about spike train generation. Second, information measures can be used
as model selection tools for analyzing spike train processes.Comment: 20 pages, 6 figures;
http://csc.ucdavis.edu/~cmg/compmech/pubs/trdctim.ht
Inferring the Residual Waiting Time for Binary Stationary Time Series
summary:For a binary stationary time series define to be the number of consecutive ones up to the first zero encountered after time , and consider the problem of estimating the conditional distribution and conditional expectation of after one has observed the first outputs. We present a sequence of stopping times and universal estimators for these quantities which are pointwise consistent for all ergodic binary stationary processes. In case the process is a renewal process with zero the renewal state the stopping times along which we estimate have density one
Thermodynamics of the Binary Symmetric Channel
We study a hidden Markov process which is the result of a transmission of the
binary symmetric Markov source over the memoryless binary symmetric channel.
This process has been studied extensively in Information Theory and is often
used as a benchmark case for the so-called denoising algorithms. Exploiting the
link between this process and the 1D Random Field Ising Model (RFIM), we are
able to identify the Gibbs potential of the resulting Hidden Markov process.
Moreover, we obtain a stronger bound on the memory decay rate. We conclude with
a discussion on implications of our results for the development of denoising
algorithms
Inferring the residual waiting time for binary stationary time series
summary:For a binary stationary time series define to be the number of consecutive ones up to the first zero encountered after time , and consider the problem of estimating the conditional distribution and conditional expectation of after one has observed the first outputs. We present a sequence of stopping times and universal estimators for these quantities which are pointwise consistent for all ergodic binary stationary processes. In case the process is a renewal process with zero the renewal state the stopping times along which we estimate have density one
Estimating the entropy of binary time series: Methodology, some theory and a simulation study
Partly motivated by entropy-estimation problems in neuroscience, we present a
detailed and extensive comparison between some of the most popular and
effective entropy estimation methods used in practice: The plug-in method, four
different estimators based on the Lempel-Ziv (LZ) family of data compression
algorithms, an estimator based on the Context-Tree Weighting (CTW) method, and
the renewal entropy estimator.
**Methodology. Three new entropy estimators are introduced. For two of the
four LZ-based estimators, a bootstrap procedure is described for evaluating
their standard error, and a practical rule of thumb is heuristically derived
for selecting the values of their parameters. ** Theory. We prove that, unlike
their earlier versions, the two new LZ-based estimators are consistent for
every finite-valued, stationary and ergodic process. An effective method is
derived for the accurate approximation of the entropy rate of a finite-state
HMM with known distribution. Heuristic calculations are presented and
approximate formulas are derived for evaluating the bias and the standard error
of each estimator. ** Simulation. All estimators are applied to a wide range of
data generated by numerous different processes with varying degrees of
dependence and memory. Some conclusions drawn from these experiments include:
(i) For all estimators considered, the main source of error is the bias. (ii)
The CTW method is repeatedly and consistently seen to provide the most accurate
results. (iii) The performance of the LZ-based estimators is often comparable
to that of the plug-in method. (iv) The main drawback of the plug-in method is
its computational inefficiency.Comment: 34 pages, 3 figure
- …