Forward and reverse representations for Markov chains

In this paper we carry over the concept of reverse probabilistic representations developed in Milstein, Schoenmakers, Spokoiny (2004) for diffusion processes, to discrete time Markov chains. We outline the construction of reverse chains in several situations and apply this to processes which are connected with jump-diffusion models and finite state Markov chains. By combining forward and reverse representations we then construct transition density estimators for chains which have root-N accuracy in any dimension and consider some applications

Forward and reverse representations for Markov chains

In this paper we carry over the concept of reverse probabilistic representations developed in Milstein, Schoenmakers, Spokoiny (2004) for diffusion processes, to discrete time Markov chains. We outline the construction of reverse chains in several situations and apply this to processes which are connected with jump-diffusion models and finite state Markov chains. By combining forward and reverse representations we then construct transition density estimators for chains which have root-N accuracy in any dimension and consider some applications

Forward and reverse representations for Markov chains

In this paper we carry over the concept of reverse probabilistic representations developed in Milstein, Schoenmakers, Spokoiny (2004) for diffusion processes, to discrete time Markov chains. We outline the construction of reverse chains in several situations and apply this to processes which are connected with jump-diffusion models and finite state Markov chains. By combining forward and reverse representations we then construct transition density estimators for chains which have root-N accuracy in any dimension and consider some applications

Optimal Estimation via Nonanticipative Rate Distortion Function and Applications to Time-Varying Gauss-Markov Processes

In this paper, we develop {finite-time horizon} causal filters using the nonanticipative rate distortion theory. We apply the {developed} theory to {design optimal filters for} time-varying multidimensional Gauss-Markov processes, subject to a mean square error fidelity constraint. We show that such filters are equivalent to the design of an optimal \texttt{\{encoder, channel, decoder\}}, which ensures that the error satisfies {a} fidelity constraint. Moreover, we derive a universal lower bound on the mean square error of any estimator of time-varying multidimensional Gauss-Markov processes in terms of conditional mutual information. Unlike classical Kalman filters, the filter developed is characterized by a reverse-waterfilling algorithm, which ensures {that} the fidelity constraint is satisfied. The theoretical results are demonstrated via illustrative examples.Comment: 35 pages, 6 figures, submitted for publication in SIAM Journal on Control and Optimization (SICON

Information Symmetries in Irreversible Processes

We study dynamical reversibility in stationary stochastic processes from an information theoretic perspective. Extending earlier work on the reversibility of Markov chains, we focus on finitary processes with arbitrarily long conditional correlations. In particular, we examine stationary processes represented or generated by edge-emitting, finite-state hidden Markov models. Surprisingly, we find pervasive temporal asymmetries in the statistics of such stationary processes with the consequence that the computational resources necessary to generate a process in the forward and reverse temporal directions are generally not the same. In fact, an exhaustive survey indicates that most stationary processes are irreversible. We study the ensuing relations between model topology in different representations, the process's statistical properties, and its reversibility in detail. A process's temporal asymmetry is efficiently captured using two canonical unifilar representations of the generating model, the forward-time and reverse-time epsilon-machines. We analyze example irreversible processes whose epsilon-machine presentations change size under time reversal, including one which has a finite number of recurrent causal states in one direction, but an infinite number in the opposite. From the forward-time and reverse-time epsilon-machines, we are able to construct a symmetrized, but nonunifilar, generator of a process---the bidirectional machine. Using the bidirectional machine, we show how to directly calculate a process's fundamental information properties, many of which are otherwise only poorly approximated via process samples. The tools we introduce and the insights we offer provide a better understanding of the many facets of reversibility and irreversibility in stochastic processes.Comment: 32 pages, 17 figures, 2 tables; http://csc.ucdavis.edu/~cmg/compmech/pubs/pratisp2.ht

Back to the Future: Generating Moment Implications for Continuous-Time Markov Processes

Continuous-time Markov processes can be characterized conveniently by their infinitesimal generators. For such processes there exist forward and reverse-time generators. We show how to use these generators to construct moment conditions implied by stationary Markov processes. Generalized method of moments estimators and tests can be constructed using these moment conditions. The resulting econometric methods are designed to be applied to discrete-time data obtained by sampling continuous-time Markov processes.

Stochastic calculus over symmetric Markov processes without time reversal

We refine stochastic calculus for symmetric Markov processes without using time reverse operators. Under some conditions on the jump functions of locally square integrable martingale additive functionals, we extend Nakao's divergence-like continuous additive functional of zero energy and the stochastic integral with respect to it under the law for quasi-everywhere starting points, which are refinements of the previous results under the law for almost everywhere starting points. This refinement of stochastic calculus enables us to establish a generalized Fukushima decomposition for a certain class of functions locally in the domain of Dirichlet form and a generalized It\^{o} formula. (With Errata.)Comment: Published in at http://dx.doi.org/10.1214/09-AOP516 and Errata at http://dx.doi.org/10.1214/11-AOP700 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org