72,837 research outputs found
Approximation of Markov Processes by Lower Dimensional Processes via Total Variation Metrics
The aim of this paper is to approximate a finite-state Markov process by
another process with fewer states, called herein the approximating process. The
approximation problem is formulated using two different methods.
The first method, utilizes the total variation distance to discriminate the
transition probabilities of a high dimensional Markov process and a reduced
order Markov process. The approximation is obtained by optimizing a linear
functional defined in terms of transition probabilities of the reduced order
Markov process over a total variation distance constraint. The transition
probabilities of the approximated Markov process are given by a water-filling
solution.
The second method, utilizes total variation distance to discriminate the
invariant probability of a Markov process and that of the approximating
process. The approximation is obtained via two alternative formulations: (a)
maximizing a functional of the occupancy distribution of the Markov process,
and (b) maximizing the entropy of the approximating process invariant
probability. For both formulations, once the reduced invariant probability is
obtained, which does not correspond to a Markov process, a further
approximation by a Markov process is proposed which minimizes the
Kullback-Leibler divergence. These approximations are given by water-filling
solutions.
Finally, the theoretical results of both methods are applied to specific
examples to illustrate the methodology, and the water-filling behavior of the
approximations.Comment: 38 pages, 17 figures, submitted to IEEE-TA
Diffusion Approximations for Online Principal Component Estimation and Global Convergence
In this paper, we propose to adopt the diffusion approximation tools to study
the dynamics of Oja's iteration which is an online stochastic gradient descent
method for the principal component analysis. Oja's iteration maintains a
running estimate of the true principal component from streaming data and enjoys
less temporal and spatial complexities. We show that the Oja's iteration for
the top eigenvector generates a continuous-state discrete-time Markov chain
over the unit sphere. We characterize the Oja's iteration in three phases using
diffusion approximation and weak convergence tools. Our three-phase analysis
further provides a finite-sample error bound for the running estimate, which
matches the minimax information lower bound for principal component analysis
under the additional assumption of bounded samples.Comment: Appeared in NIPS 201
Approximations of countably-infinite linear programs over bounded measure spaces
We study a class of countably-infinite-dimensional linear programs (CILPs)
whose feasible sets are bounded subsets of appropriately defined weighted
spaces of measures. We show how to approximate the optimal value, optimal
points, and minimal points of these CILPs by solving finite-dimensional linear
programs. The errors of our approximations converge to zero as the size of the
finite-dimensional program approaches that of the original problem and are easy
to bound in practice. We discuss the use of our methods in the computation of
the stationary distributions, occupation measures, and exit distributions of
Markov~chains
Quantitative Approximation of the Probability Distribution of a Markov Process by Formal Abstractions
The goal of this work is to formally abstract a Markov process evolving in
discrete time over a general state space as a finite-state Markov chain, with
the objective of precisely approximating its state probability distribution in
time, which allows for its approximate, faster computation by that of the
Markov chain. The approach is based on formal abstractions and employs an
arbitrary finite partition of the state space of the Markov process, and the
computation of average transition probabilities between partition sets. The
abstraction technique is formal, in that it comes with guarantees on the
introduced approximation that depend on the diameters of the partitions: as
such, they can be tuned at will. Further in the case of Markov processes with
unbounded state spaces, a procedure for precisely truncating the state space
within a compact set is provided, together with an error bound that depends on
the asymptotic properties of the transition kernel of the original process. The
overall abstraction algorithm, which practically hinges on piecewise constant
approximations of the density functions of the Markov process, is extended to
higher-order function approximations: these can lead to improved error bounds
and associated lower computational requirements. The approach is practically
tested to compute probabilistic invariance of the Markov process under study,
and is compared to a known alternative approach from the literature.Comment: 29 pages, Journal of Logical Methods in Computer Scienc
From Infinite to Finite Programs: Explicit Error Bounds with Applications to Approximate Dynamic Programming
We consider linear programming (LP) problems in infinite dimensional spaces
that are in general computationally intractable. Under suitable assumptions, we
develop an approximation bridge from the infinite-dimensional LP to tractable
finite convex programs in which the performance of the approximation is
quantified explicitly. To this end, we adopt the recent developments in two
areas of randomized optimization and first order methods, leading to a priori
as well as a posterior performance guarantees. We illustrate the generality and
implications of our theoretical results in the special case of the long-run
average cost and discounted cost optimal control problems for Markov decision
processes on Borel spaces. The applicability of the theoretical results is
demonstrated through a constrained linear quadratic optimal control problem and
a fisheries management problem.Comment: 30 pages, 5 figure
Laguerre and Meixner symmetric functions, and infinite-dimensional diffusion processes
The Laguerre symmetric functions introduced in the note are indexed by
arbitrary partitions and depend on two continuous parameters. The top degree
homogeneous component of every Laguerre symmetric function coincides with the
Schur function with the same index. Thus, the Laguerre symmetric functions form
a two-parameter family of inhomogeneous bases in the algebra of symmetric
functions. These new symmetric functions are obtained from the N-variate
symmetric polynomials of the same name by a procedure of analytic continuation.
The Laguerre symmetric functions are eigenvectors of a second order
differential operator, which depends on the same two parameters and serves as
the infinitesimal generator of an infinite-dimensional diffusion process X(t).
The process X(t) admits approximation by some jump processes related to one
more new family of symmetric functions, the Meixner symmetric functions. In
equilibrium, the process X(t) can be interpreted as a time-dependent point
process on the punctured real line R\{0}, and the point configurations may be
interpreted as doubly infinite collections of particles of two opposite charges
with log-gas-type interaction. The dynamical correlation functions of the
equilibrium process have determinantal form: they are given by minors of the
so-called extended Whittaker kernel, introduced earlier in a paper by Borodin
and the author.Comment: LaTex, 26 p
The non-locality of Markov chain approximations to two-dimensional diffusions
In this short paper, we consider discrete-time Markov chains on lattices as
approximations to continuous-time diffusion processes. The approximations can
be interpreted as finite difference schemes for the generator of the process.
We derive conditions on the diffusion coefficients which permit transition
probabilities to match locally first and second moments. We derive a novel
formula which expresses how the matching becomes more difficult for larger
(absolute) correlations and strongly anisotropic processes, such that
instantaneous moves to more distant neighbours on the lattice have to be
allowed. Roughly speaking, for non-zero correlations, the distance covered in
one timestep is proportional to the ratio of volatilities in the two
directions. We discuss the implications to Markov decision processes and the
convergence analysis of approximations to Hamilton-Jacobi-Bellman equations in
the Barles-Souganidis framework.Comment: Corrected two errata from previous and journal version: definition of
R in (5) and summations in (7
A method for pricing American options using semi-infinite linear programming
We introduce a new approach for the numerical pricing of American options.
The main idea is to choose a finite number of suitable excessive functions
(randomly) and to find the smallest majorant of the gain function in the span
of these functions. The resulting problem is a linear semi-infinite programming
problem, that can be solved using standard algorithms. This leads to good upper
bounds for the original problem. For our algorithms no discretization of space
and time and no simulation is necessary. Furthermore it is applicable even for
high-dimensional problems. The algorithm provides an approximation of the value
not only for one starting point, but for the complete value function on the
continuation set, so that the optimal exercise region and e.g. the Greeks can
be calculated. We apply the algorithm to (one- and) multidimensional diffusions
and to L\'evy processes, and show it to be fast and accurate
- …